Medical image processing system, medical image processing apparatus, control method thereof, and recording medium

ABSTRACT

A medical image processing system that can accept commissions to perform 3D formation using a 3D forming apparatus includes a storage unit, a selection accepting unit, a part identifying unit, and a screen generating unit. The storage unit stores a medical image, and body part information representing a human body part in the medical image. The selection accepting unit accepts selection with regard to the medical image stored in the storage unit. The part identifying unit identifies the body part included in the medical image, using the body part information corresponding to the medical image regarding which the selection accepting unit has accepted selection. The screen generating unit generates a screen that can accept a commission to form a formed object of the body part identified by the part identifying unit.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The disclosure in the present specification relates to a medical imageprocessing system, a medical image processing apparatus, a controlmethod thereof, and a recording medium.

2. Description of the Related Art

There is a mechanism where multiple medical images, generated bymodalities such as X-ray computed tomography (CT) or magnetic resonanceimaging (MRI), are used to generate volume data on a computer, andvolume rendering is performed to three-dimensionally display parts ofthe human body.

This mechanism is realized by a computer (or application) called amedical 3D workstation. In other words, a user needs to have such anenvironment to perform 3D display of human body parts using medicalimages. Accordingly, 3D data is generated in formats such as virtualreality modeling language (VRML) format or STereoLithography (STL)format files representing parts of the human body, and displayed. Thus,3D display of parts of the human body can be performed even without theabove-described environment.

Japanese Patent Laid-Open No. 2013-138763 discloses a mechanism where acontour of a bone is extracted from voxel data generated from multiplemedical images, and output as 3D data in the STL format. There also is amechanism where human body parts are not only displayed in 3D in avirtual manner on a computer, but also output as formed objects formedby a 3D forming apparatus. These formed objects are used in the medicalfield and in research. Outputting such formed objects from a 3D formingapparatus requires knowledge regarding medicine, knowledge regardingimage processing of medical images, and further knowledge regarding 3Dforming apparatus, and accordingly is not readily accessible.

A mechanism disclosed in Japanese Patent Laid-Open No. 2002-86576 had aproblem in that a user commissioning a service business to form such aformed object needed to know beforehand what sort of formed objects canbe formed from medical images sent to the service business. In otherwords, the commissioning side had to recognize what part of the humanbody a cross-section medical image being sent shows, and what organs(body parts) are included in that cross-section. Otherwise, there is thepossibility that an order might be placed to form an organ that cannotbe formed from the medical image. Accordingly, Japanese Patent Laid-OpenNo. 2002-86576 discloses a mechanism where a formed object of anobjective body part can be formed simply by transmitting to the systemof the service business medical images, the medical images, the bodypart which the commissioning part wants output as a formed object, andother instructions necessary for the forming.

As for another problem, the mechanism in Japanese Patent Laid-Open No.2002-86576 permitted the commissioning user to commission the sameformation time after time, leading to a greater processing load on theserver at the service business. The service business server performsimage processing to extract human body parts from medical images. Volumedata needs to be generated to performing forming from medical images atthe 3D forming apparatus, and the processing load of extracting aparticular part from the generated volume data is greater than withtwo-dimensional image processing. That is to say, each time thecommissioning user forgets that a formation was commission and placesthe same order again, the server has to perform the image processingjust that many times, leading to an increased processing load.

As for another problem, the mechanism in Japanese Patent Laid-Open No.2013-138763 requires the above-described environment to generatethree-dimensional data, after all. That is to say, users who had fewopportunities to perform 3D display of human body parts have found itdifficult to casually view 3D data. Even if the user were able toprepare the above-described environment, it would be difficult toextract a desired body part unless the user has skills for operating theapplication and sufficient knowledge of image processing for executionon medical images.

SUMMARY OF THE INVENTION

According to some embodiments of the present invention, a medical imageprocessing system that can accept commissions to perform 3D formationusing a 3D forming apparatus includes a storage unit configured to storea medical image, and body part information representing a human bodypart in the medical image, a selection accepting unit configured toaccept selection with regard to the medical image stored in the storageunit, a part identifying unit configured to identify the body partincluded in the medical image, using the body part informationcorresponding to the medical image regarding which the selectionaccepting unit has accepted selection, and a screen generating unitconfigured to generate a screen that can accept a commission to form aformed object of the body part identified by the part identifying unit.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an overview of the present invention.

FIG. 2 is a diagram illustrating the configuration of a medical imageprocessing system.

FIG. 3 is a diagram illustrating the hardware configuration of a clientterminal and a medical image server.

FIG. 4 is a diagram illustrating the functional configuration of theclient terminal and the medical image processing server.

FIG. 5 is a flowchart diagram illustrating a series of processing fordisplaying a medical image list screen.

FIG. 6 is a diagram illustrating the table configuration of a medicalimage management table.

FIG. 7 is a diagram illustrating a screen configuration of the medicalimage list screen.

FIG. 8 is a flowchart diagram illustrating a series of processing fordisplaying a commissioning content input screen.

FIG. 9 is a diagram illustrating the table configuration of a formationpart management table.

FIG. 10 is a diagram illustrating a screen configuration of thecommissioning content input screen.

FIG. 11 is a flowchart diagram illustrating a series of processing foraccepting commission content.

FIG. 12 is a diagram illustrating the table configuration of acommission information management table.

FIG. 13 is a diagram illustrating the overview of generating 3D PDF datafrom medical images.

FIG. 14 is a flowchart diagram illustrating a series of processing fordisplaying commission results.

FIG. 15 is a diagram illustrating 3D PDF data displayed.

FIG. 16 is a flowchart diagram illustrating a series of processing forfinalizing commission.

FIG. 17 is a flowchart diagram illustrating a series of processing forcancelling a commission.

FIG. 18 is a diagram illustrating an overview of the present invention.

FIG. 19 is a diagram illustrating the configuration of a medical imageprocessing system.

FIG. 20 is a diagram illustrating the hardware configuration of a clientterminal and a medical image server.

FIG. 21 is a diagram illustrating the functional configuration of theclient terminal and the medical image server.

FIG. 22 is a flowchart diagram illustrating a series of processing fordisplaying a medical image list screen.

FIG. 23 is a diagram illustrating the table configuration of a medicalimage management table.

FIG. 24 is a diagram illustrating a screen configuration of the medicalimage list screen.

FIG. 25 is a flowchart diagram illustrating a series of processing fordisplaying a commissioning content input screen.

FIG. 26 is a diagram illustrating the table configuration of a displaypart management table.

FIG. 27 is a diagram illustrating a table configuration of a displayorientation management table screen.

FIG. 28 is a diagram illustrating a screen configuration of a commissioncontent input screen.

FIG. 29 is a flowchart diagram illustrating a series of processingexecuted in a case where selection of a display part has been acceptedat the commission content input screen.

FIG. 30 is a diagram illustrating an example having accepted selectionof a display part and display orientation in the commission contentinput screen.

FIG. 31 is a flowchart diagram illustrating a series of processing ofaccepting commission content.

FIG. 32 is a diagram illustrating the table configuration of acommission information management table.

FIG. 33 is a diagram illustrating the overview of generating 3D PDF datafrom medical images.

FIGS. 34A and 34B are diagrams illustrating 3D PDF data displayed.

FIG. 35 is a flowchart diagram illustrating a series of processing ofdisplaying commission results.

FIGS. 36A and 36B are diagrams illustrating screen configurations of acommission results confirmation screen.

FIG. 37 is a flowchart diagram illustrating a series of processingexecuted in response to an operation accepted at the commission resultsconfiguration screen.

FIG. 38 is a flowchart diagram illustrating a series of processing offinalizing commissioning.

FIG. 39 is a flowchart diagram illustrating a series of processingexecuted in response to an operation accepted at the 3D PDF data.

FIG. 40 is a flowchart diagram illustrating a series of processing ofcancelling commissioning.

DESCRIPTION OF THE EMBODIMENTS

An embodiment of the present invention will be described in detail withreference to the drawings. First, an overview of the present inventionwill be described with reference to FIG. 1. A medical image processingsystem 100 according to the present invention is a system including aclient terminal 101, a medical image processing server 102, and a 3Dforming apparatus 103. The client terminal 101 and medical imageprocessing server 102 are communicably connected, and further, themedical image processing server 102 and 3D forming apparatus 103 arecommunicably connected.

The client terminal 101 transmits medical images generated by modalitiessuch as X-ray CT or MRI to the medical image processing server 102beforehand. The medical image processing server 102 stores these medicalimages. The term “medical image” in the present embodiment includestomographic images generated by one modality or another. The term“medical image” also includes images according to the Digital Imagingand COmmunication in Medicine (DICOM) standard. Accessory informationfor DICOM includes information of examination date, patient name, typeof examination, part or site name, and so forth. That is to say, amedical image includes this information. When storing medical images inthe medical image processing server 102, multiple images generated in asingle examination are made into a series (grouped) and stored. Notethat in the present embodiment, this medical image group that has beenmade into a series will be referred to as “medical image”.

In a case where a user of the client terminal 101 wants to form a formedobject representing a human body part that is included in a medicalimage stored in the medical image processing server 102, using a 3Dforming apparatus, the client terminal 101 accepts instructions to thateffect. A screen such as indicated by reference numeral 110 in FIG. 1 isdisplayed on a display of the client terminal 101. At the time ofgenerating this image at the medical image processing server 102, a“commission” button for medical images that have already beencommissioned is hidden so that such a medical image is not commissionedagain. Thus, the commissioning user can identify that a medical imagehas already be commissioned. This reduces the processing load at themedical image processing server 102, since there are no redundantcommissions for formation.

Next, the commissioning user presses the commission button provided tothe relevant record of the medical image. Upon detecting this pressing,a screen such as indicated by reference numeral 120 in FIG. 1 isdisplayed on the display of the client terminal 101. This screen acceptsselection of the human body part that the user wants to form.

Upon the user selecting the part to form, and the client terminal 101detects pressing of an OK button, a formation commission is transmittedform the client terminal 101 to the medical image processing server 102.The medical image processing server 102 accepts this commission, andgenerates volume data using the relevant medical image. Volume data isdata where medical images are layered three-dimensionally, and a CTnumber (a relative value where water is 0 and air is −1000) that eachpixel in the medical image has is given as a parameter to a voxelcorresponding to that pixel. The medical image processing server 102applies visual information (color, viewpoint, slight source, gloss,etc.) to the volume data, thereby generating 3D data. This 3D data isdata representing a 3D model in the VRML format or STL format, forexample.

This sort of data that can be formed at the 3D forming apparatus 103(hereinafter referred to as forming job) is generated by a driver(hereinafter referred to as 3D forming driver, the forming job istransmitted to the 3D forming apparatus 103, and a formed object isoutput. In this mechanism where formation of human body parts using amedical image is commissioned and forming is performed at a 3D formingapparatus, hiding the commission button for medical images that havealready been commissioned realizes a screen where redundant formationcommissioning cannot be accepted, and the user cannot perform redundantformation commissioning. This overview will be described in furtherdetail.

FIG. 2 is a diagram illustrating the system configuration of the medicalimage processing system 100. Note that the system configurationillustrated in FIG. 2 is only exemplary. The medical image processingsystem 100 is a system including the client terminal 101, medical imageprocessing server 102, and 3D forming apparatus 103. The client terminal101 and medical image processing server 102, and the medical imageprocessing server 102 and 3D forming apparatus 103, are eachcommunicably connected via a local area network (LAN) 201 or theInternet or the like. An arrangement may also be made where the 3Dforming apparatus 103 is not included in the medical image processingsystem 100.

The client terminal 101 is one or multiple devices at the side of thecommissioning user that commissions 3D formation. The client terminal101 may be a personal computer, or may be a mobile terminal (cellularphone, smartphone, wearable device, etc.) or the like.

The medical image processing server 102 (medical image processingapparatus) is one or multiple devices at the side of a business thataccepts commissions for 3D forming. Although a server device is assumedas the medical image processing server 102, any device capable ofprocessing commissions from the client terminal 101 may be used.

The 3D forming apparatus 103 is one or multiple apparatuses that output3D formed objects (a so-called 3D printer). A business may own the 3Dforming apparatus 103, or a different arrangement may be used. Themethod of formation that the 3D forming apparatus 103 uses may beadditive layer manufacturing, ultrasonic additive manufacturing, or anyother of the many methods that exist. The material of the object to beformed may be resin, metal, rubber, or any other of the many materialthat exist, and is not restricted in particular. The medical imageprocessing server 102 preferably is communicably connected with varioustypes of 3D forming apparatuses 103, to give the commissioning party awider range of options when commissioning.

FIG. 3 is a diagram that illustrates the hardware configuration of theclient terminal 101 and medical image processing server 102. Note thatthe hardware configuration in FIG. 3 is only exemplary. A centralprocessing unit (CPU) 301 centrally controls various devices andcontrollers connected to a system bus 304.

Read-only memory (ROM) 302 or external memory 311 store a basicinput/output system (BIOS) and an operating system program, which arecontrol programs for the CPU 301. The external memory 311 also storesprograms that are necessary for the devices to realize the functionsthat they execute. Random access memory (RAM) 303 functions as mainmemory, work area, and so forth, for the CPU 301.

The CPU 301 loads programs and the like necessary to execute processingto the RAM 303, and executes the programs, thereby realizing the varioustypes of operations. An input controller 305 controls input from aninput device 309 such as a keyboard or a pointing device like a mouse orthe like. A video controller 306 controls display on a display devicesuch as a display 310 or the like. The display device may be a cathoderadio tube (CRT), liquid crystal display, or the like.

A memory controller 307 controls access to the external memory 311.Examples of the external memory 311 include a hard disk, flexible disk,card-type memory connected to a Personal Computer Memory CardInternational Association (PCMCIA) card slot via an adapter, and soforth. The external memory 311 (storage unit) stores a boot program,browser software, various types of applications, font data, user files,editing files, various types of data, and so forth.

A communication interface controller 308 is for connection to andcommunication with external devices via a network, and executescommunication control processing on the network. For example, Internetcommunication using TCP/IP can be performed.

The CPU 301 also enables display to be made on the display 310 byrendering (rasterizing) outline fonts to a display information area inthe RAM 303. The CPU 301 further also enables user instructions on thedisplay 310 by a mouse cursor or the like, omitted from illustration.

The various types of programs and the like used for the client terminal101 and medical image processing server 102 to execute various types ofprocessing that will be described later, are recorded in the externalmemory 311. These various types of programs and so forth are loaded tothe RAM 303 as necessary, and thus are executed by the CPU 301. Further,definition files various types of information tables used by programsaccording to the present invention are stored in the external memory311.

FIG. 4 is a diagram illustrating an example of the functionalconfiguration of the client terminal 101 and medical image processingserver 102. The functions illustrated in FIG. 4 are the componentsrealized by the hardware arrangements, various types of programs, and soforth, illustrated in FIG. 2. Note that the functional configurationillustrated in FIG. 4 is only exemplary. The client terminal 101includes a communication control unit 401, a display unit 402, and a Webbrowser unit 403, as functional units.

The communication control unit 401 is a functional unit for exchangingvarious types of information with the medical image processing server102. The communication control unit 401 communicates with the medicalimage processing server 102 on the network via the communicationinterface controller 308 of the client terminal 101.

The display unit 402 is a functional unit for displaying various typesof information. The display unit 402 transmits images including varioustypes of information to the display 310 via the video controller 306 ofthe client terminal 101, so as to display the various types ofinformation on the display 310.

The Web browser unit 403 is a functional unit for displaying andoperating Web pages (referred to as “screens” in the presentembodiment), described in HyperText Markup Language (HTML). The Webbrowser unit 403 further includes a rendering unit 404 and an operationaccepting unit 405.

The rendering unit 404 is a functional unit that analyzes the contentsdescribed in the Web page, and displays characters and images on thedisplay. The operation accepting unit 405 is a functional unit thataccepts operations made regarding the Web page. The operation acceptingunit 405 also transmits acquisition requests for Web pages and files tothe medical image processing server 102 via the communication controlunit 401, in accordance with accepted operations.

The medical image processing server 102 includes, as functional units, acommunication control unit 411, a storage unit 412, a screen generatingunit 413, a 3D forming driver unit 415, and a 3D control unit 417.

The communication control unit 411 is a functional unit that exchangesvarious types of information with the client terminal 101 and 3D formingapparatus 103. The communication control unit 411 communicates with theclient terminal 101 and 3D forming apparatus 103 on the network via thecommunication interface controller 308 of the medical image processingserver 102.

The storage unit 412 is a functional unit that stores various types ofinformation in the RAM 303 or external memory 311. Examples of thevarious types of information that the storage unit 412 stores includesvarious types of tables that will be described later, templates for Webpages to be displayed at the client terminal 101, and so forth.

The screen generating unit 413 is a functional unit that generatesscreens (Web pages) to be displayed on the client terminal 101. Thescreen generating unit 413 generates screens using the various types ofinformation stored in the storage unit 412. The screen generating unit413 further includes a formation part identifying unit 414.

The formation part identifying unit 414 is functional unit that acquiresmedical images instructed by the client terminal 101 from the storageunit 412, and identifies human body parts that can be formed from themedical images. The formation part identifying unit 414 identifiesformable human body parts from information of the examined partsincluded in the DICOM information of the medical images.

The 3D forming driver unit 415 is a functional unit that gives formationinstructions to the 3D forming apparatus 103. The 3D forming driver unit415 further includes a formation job generating unit 416.

The formation job generating unit 416 is a functional unit thatgenerates a formation job for formation of the 3D data generated by alater-described 3D data generating unit 420, using the 3D formingapparatus 103. The method of generating a formation job for formation atthe 3D forming apparatus 103 is conventional art, and will be describedlater.

The 3D control unit 417 is a functional unit for performing control suchas generating and displaying three-dimensional data in three-dimensionalspace. The 3D control unit 417 further includes a volume data generatingunit 418, a formation part extracting unit 419, the 3D data generatingunit 420, and a 3D PDF data generating unit 421.

The volume data generating unit 418 is a functional unit that generatesvolume data from medical images. The volume data generating unit 418generates volume data using multiple medical images. In a case wheremedical images to generate volume data are insufficient, the volume datagenerating unit 418 interpolates for insufficient portions. The methodof generating volume data is conventional art, and will be describedlater.

The formation part extracting unit 419 is a functional unit thatextracts the human body part selected at the client terminal 101 fromthe volume data. The storage unit 412 stores a known algorithm forextracting each human body part, and the formation part extracting unit419 selects the necessary algorithm from the storage unit 412 to performextracting. The method of extracting particular body parts from volumedata, and the algorithms thereof, also are conventional art, and will bedescribed later.

The 3D data generating unit 420 is a functional unit that generates 3Ddata using the volume data from which the formation part extracting unit419 has extracted a particular body part. The 3D data generating unit420 generates 3D data in formats that the 3D forming driver unit 415 cananalyze, such as the VRML and STL formats.

The 3D PDF data generating unit 421 is a functional unit that generatesPortable Document Format (PDF) data including 3D data (hereinafterreferred to as 3D PDF data). The 3D PDF data is PDF data where the usercan view 3D data from any viewpoint. The method of generating the 3D PDFdata also is conventional art, and will be described later.

FIG. 5 is a diagram illustrating a series of processing for displaying amedical image list screen 700. The steps of step S501, step S509, andstep S510 are processing executed by the CPU 301 of the client terminal101. The steps of step S502 through step S508 in FIG. 5 are processingexecuted by CPU 301 of the medical image processing server 102. Notethat the contents of processing and the order of processing illustratedin FIG. 5 are only exemplary, and are not restrictive.

In step S501, the communication control unit 411 of the client terminal101 transmits an acquisition request for a medical image list screen tothe medical image processing server 102.

In step S502, the communication control unit 411 of the medical imageprocessing server 102 receives the acquisition request for the medicalimage list screen transmitted from the client terminal 101. In stepS503, the screen generating unit 413 of the medical image processingserver 102 acquires a template for a medical image list screen from thestorage unit 412 and information stored in a medical image managementtable 600 illustrated in FIG. 6, and generates a medical image listscreen 700 such as illustrated in FIG. 7. The medical image list screen700 includes a medical image list 701, with a record being created inthe medical image list 701 for each record in the medical imagemanagement table 600. A corresponding image ID 601 is embedded in eachrecord in the medical image list 701.

When creating the screen in step S502, none of a commission button 702,confirm button 703, finalize button 704 and cancel button 705, which arelaid out in later-described processing, are disposed yet. Further, themedical image list 701 includes a display button 706 for each medicalimage record. The display button 706 is for displaying the medicalimage. Accordingly, the display button 706 is disposed as a button toacquire the medical image corresponding to the record where the displaybutton 706 is disposed from a medical image saving location 607, anddisplaying the medical image.

The medical image management table 600 is a table for storinginformation relating to medical images stored in the external memory 311of the medical image processing server 102. The medical image managementtable 600 is stored in the external memory 311 of the medical imageprocessing server 102. Upon receiving a medical image from the clientterminal 101 or an apparatus of one modality or another, a new record iscreated in the medical image management table 600, various types ofinformation are acquired from the medical image DICOM tag, and stored inthe medical image management table 600. Note that the tableconfiguration of the medical image management table 600 illustrated hereis only exemplary, and is not restrictive.

Items of the medical image management table 600 are image ID 601,examination date 602, patient name 603, examination type 604, site 605,number 606, and medical image saving location 607. The image ID 601 isan item storing identification information that is uniquely allocated toeach medical image that has been received. The examination date 602 isan item storing the date on which the medical image was imaged by anapparatus of one modality or another. The patient name 603 is an itemstoring the full name of the patient. The site 605 is an item storingthe site of imaging the medical image. The number 606 is an item storingthe number of received medical images that are stored. The medical imagesaving location 607 is an item sorting information indicating a folderwhere the medical images are stored.

In step S504, the screen generating unit 413 of the medical imageprocessing server 102 references one of the records in the medical imagemanagement table 600. Upon having completed the steps of step S505through step S508 regarding the referenced record, determination is maderegarding whether or not the processing of step S505 through step S508has been completed for all records in the medical image management table600. If determination is made that the processing has been completed,the flow advances to step S509. That is to say, step S505 through stepS508 are looped through for all records in the medical image managementtable 600. The individual steps in step S505 through step S508 will bedescribed next.

In step S505, the screen generating unit 413 of the medical imageprocessing server 102 determines whether the number of medical images ofthe record being referenced in the number 606 is a predetermined numberor more, and also that commission information of the medical images doesnot exist in a commission information management table 1200 illustratedin FIG. 12. If both are true, the flow advances to step S506, andotherwise, the flow advances to step S507. Determination may also bemade regarding whether the number of medical images is larger than thepredetermined number, and also that no commission information exists.The number of medical images is set beforehand, to a number from whichvolume data indicating body parts can be generated from medical images.Alternatively, the number may be based on a number that will keepcoarseness in the generated volume data to an inconspicuous level, or anumber where the output formed object will have smooth curved surfaces.

The commission information management table 1200 is a data table storinginformation relating to formation commissions. One record of thecommission information management table 1200 is referred to as“commission information” in the present embodiment. Details of thecommission information management table 1200 will be described later.

In step S506, the screen generating unit 413 of the medical imageprocessing server 102 disposes the commission button 702 (acceptingportion) to the record of the medical image list 701 corresponding tothe record being referenced. The commission button 702 is a button thatcan accept a formation commission (formation instructions) of human bodyparts from a medical image. A medical image that has a sufficient numberis a medical image regarding which a formation commission can be made,so the commission button 702 is disposed thereat. Further, in a casewhere a formation commission can be made but a formation commission hasalready been made, a redundant commission raises the processing load onthe medical image processing server 102. Accordingly, in a case where aformation commission can be made but a formation commission has alreadybeen made, the commission button 702 is not provided, while thecommission button 702 is provided if no formation commission has beenmade yet. By distinguishing this display from one medical image toanother, the processing load on the medical image processing server 102can be reduced, and the user can recognize each medical image regardingwhich a formation commission has already been made. Moreover, as aresult of the determination in step S505, in a case where the number ofmedical images is less than the predetermined number (or may be equal toor less), step S506 is not executed, so no commission button 702 isprovided for the record of this medical image. Accordingly, the user canbe prevented from making formation commissions of medical images with aninsufficient number.

Although an arrangement has been described where further formationinstructions are not given for medical image records that have alreadybeen commissioned by not disposing the commission button, but anarrangement may be made where the commission button is displayed in away that the button cannot be pressed, thereby facilitating usercomprehension that the commissioning has already been performed.Alternatively, an arrangement may be made where the commission button isdisplayed in a different color or shape, or with a different textstring, thereby facilitating user comprehension that the commissioninghas already been performed. Further, an arrangement may be made wherethe commission button is disposed, and the disposed commission buttoncan be pressed, but an error is returned when the commission button ispressed and no formation commission is placed.

In step S507, the screen generating unit 413 of the medical imageprocessing server 102 determines whether or not commission informationcorresponding to the medical image of the record being referenced, andwhether or not a status 1208 of the commission information is “currentlyconfirming commissioning party”. That is to say, determination is maderegarding whether or not formation using the medical image of the recordbeing reference has already been commissioned, and commission resultscorresponding to the commission have already been output. If so, theflow advances to step S508, and otherwise, the flow is returned to stepS504 and whether or not the loop has been completed is determined. In acase where determination is made that the loop has been completed, theflow advances to step S509.

In step S508, the screen generating unit 413 of the medical imageprocessing server 102 disposes the confirm button 703, finalize button704, and cancel button 705 at the record of the commission button 702corresponding to the record being referenced in the medical image list701.

The confirm button 703 is a button for confirming the commissionresults. In a case where the confirmation format 1206 of the commissioninformation is 3D data, the confirm button 703 is a button fordownloading the 3D data of the confirmation results from a savinglocation indicated by a 3D data saving location 1209. On the other hand,in a case where the confirmation format 1206 of the commissioninformation is 3D PDF data, the confirm button 703 is a button fordisplaying 3D PDF data of the commission results from a saving locationindicated by 3D PDF data saving location 1210. That is to say, theoperations in a case where the confirm button 703 is pressed is decidedaccording to the confirmation format selected by the user, who is thecommissioning party. The display form of the confirm button 703 ischanged to enable recognition of whether a 3D data download button or a3D PDF data display button. For example, this recognition can be enabledby changing the color of the confirm button 703.

The finalize button 704 is a button for finalizing the commission. Thecancel button 705 is a button for canceling the commission. The confirmbutton 703 is thus displayed in a case where commission results areoutput, so the user who is the commissioning part can recognize for eachmedical image whether or not commission results have been output, due tothe display being identified in this way.

When step S508 is complete, the flow returns to step S504 anddetermination is made regarding whether or not the loop has beencompleted. If determination is made regarding that the loop has beencompleted, the flow advances to step S509.

In step S509, The communication control unit 411 of the medical imageprocessing server 102 transmits the medical image list screen 700generated in step S503 through step S508 to the client terminal 101.

In step S510, the communication control unit 401 of the client terminal101 receives the medical image list screen 700 transmitted from themedical image processing server 102. In step S511, the rendering unit404 of the client terminal 101 renders the received medical image listscreen 700, and the display unit 402 displays the rendering results onthe display 310 of the client terminal 101.

After displaying the medical image list screen 700, the client terminal101 executes the flowcharts illustrated in FIGS. 8, 14, 16, and 17, byparallel processing.

FIG. 8 is a flowchart diagram illustrating a series of processing fordisplaying a commissioning content input screen 1000. The steps of stepS801, step S802, step S807, and step S808 in FIG. 8 are processingexecuted by the CPU 301 of the client terminal 101. The steps of stepS803 through step S806 in FIG. 8 are processing executed by CPU 301 ofthe medical image processing server 102. Note that the contents ofprocessing and the order of processing illustrated in FIG. 8 are onlyexemplary, and are not restrictive.

In step S801, the operation accepting unit 405 of the client terminal101 determines whether or not an operation of pressing the commissionbutton 702 in the medical image list screen 700 has been accepted. In acase where determination is made that an operation of pressing thecommission button 702 has been accepted, the flow advances to step S802.In a case where determination is made that an operation of pressing thecommission button 702 has not been accepted, the flow stands by.

In step S802, the Web browser unit 403 of the client terminal 101acquires the image ID embedded in the record of the commission button702 that has accepted a pressing operation. The communication controlunit 401 of the client terminal 101 then transmits to the medical imageprocessing server 102 an acquisition request for a commission contentinput screen, for the formation commission of the medical image thatthis image ID indicates (“object medical image”) in FIG. 8.

In step S803, the communication control unit 411 of the medical imageprocessing server 102 receives the acquisition request for thecommission content input screen transmitted from the client terminal101.

In step S804, the formation part identifying unit 414 of the medicalimage processing server 102 identifies the formation part from the partof the object medical image. That is to say, parts that can be formedfrom the object medical image are identified. This will be described indetail. The site 605 and examination type 604 of the object medicalimage are referenced, and identifies records having a site 901 andexamination type 902 matching these from a formation part managementtable 900 illustrated in FIG. 9. The formation part management table 900is a data table stipulating formation parts 904 that can be formed foreach site 901. That is to say, a formation part 904 in an identifiedrecord is a part that can be formed from the medical image. Althoughdescription is made in the present embodiment that parts that can beformed from medical images are identified using such a formation partmanagement table 900, parts that can be formed may be identified byanalyzing medical images instead. That is to say, medical images may beanalyzed by image processing, and analysis made regarding what parts arein the image. A method may also be used where the medical image isformed into volume data and three-dimensionally analyzed.

The formation part management table 900 is a data table for storinginformation relating to human body parts to be formed by the 3D formingapparatus. The formation part management table 900 is stored in theexternal memory 311 of the medical image processing server 102. Notethat the configuration of the formation part management table 900 isonly exemplary, and is not restrictive.

The formation part management table 900 contains the items of site 901,examination type 902, type of organ 903, formation part 904, andextraction algorithm 905. The site 901 is an image storing informationindicating the range of the human body in which the formation part 904is included. The examination type 902 is an item storing informationindicating the modality of examination type, such as X-ray CT, MRI, orthe like. The type of organ 903 is an item storing information thatindicating the type of organ to which the formation part 904 belongs.The display position of the formation part 904 in a later-describedformation part selection space 1001 is decided in accordance with thetype of organ 903. For example, a formation part 904 is displayed foreach classification indicated by the type of organ 903. The formationpart 904 is an item storing information indicating parts that can beformed. The extraction algorithm 905 is an item storing informationindicating a known algorithm for extracting the formation part 904 fromthe medical image. This formation part management table 900 is stored inthe external memory 311 of the medical image processing server 102beforehand, by the user who manages the medical image processing server102.

In step S805, the screen generating unit 413 of the medical imageprocessing server 102 generates a commission content input screen thatcan accept selection regarding parts that can be formed, identified instep S804.

FIG. 10 illustrates an example of the commission content input screen1000. The commission content input screen 1000 has a formation partselection space 1001, a formation method section space 1002, aconformation format selection space 1003, and an OK button 1004. Theformation part selection space 1001 is a checkbox style selection spacefor the user, who is the commissioning party, to select the human bodypart to be formed. This selection space accepts selection of body partsidentified in step S804. That is to say, selection of parts notidentified in step S804 is not accepted. A selection space that onlyincludes parts identified in step S804 is preferable. This arrangementenables selection of parts that can be formed, identified from theobject medical image, to be accepted, so even if the user, who is thecommissioning party, does not have extensive knowledge of medicalimages, commissioning formation of parts than cannot be formed can beprevented.

Although description has been made that commissioning formation of partsthan cannot be formed is suppressed by not providing any selection spacefor parts not identified in S804, an arrangement may be made wherespaces for these parts are provided, but displayed as selection spacesthat cannot be selected. Alternatively, these selection spaces may bedisplayed in different colors or shapes, or with the text string struckout. Further, an arrangement may be made where these items can beselected, but the OK button 1004 cannot be pressed. For example, the OKbutton 1004 may be hidden, or the color or shape changed. Further,control may be effected so that a formation commission is not performedby returning an error in a case where the OK button 1004 is pressed.

The formation method section space 1002 is a selection space forselecting whether or not formation at the 3D forming apparatus 103 isnecessary. Whether or not formation is necessary is selected by radiobuttons, and the method by checkboxes. Selecting necessary enables theformation method to be selected. If formation unnecessary is selected,data of the format selected at the conformation format selection space1003 is generated, but formation at the 3D forming apparatus 103 is notperformed. That is to say, the user, who is the commissioning party, canperforming commissioning in order to acquire 3D data or 3D PDF data. Theconformation format selection space 1003 is a selection space to selectthe confirmation format for confirming what sort of three-dimensionalimage the human body part that has been selected by the user, who is thecommissioning party, will be. There are two types in the presentembodiment; 3D data and 3D PDF data. The OK button 1004 is a button fortransmitting the commission contents accepted at the commission contentinput screen 1000 to the medical image processing server 102. Image ID601 of the object medical image is embedded in the OK button 1004.Although omitted from illustration in the commission content inputscreen 1000 in FIG. 10, an input form for inputting commissioning partyinformation of the user, such as name, contact information (telephonenumber, address), and so forth, is also provided. The commission contentinput screen 1000 also accepts the material to be used by the 3D formingapparatus 103.

In step S806, the communication control unit 411 of the medical imageprocessing server 102 transmits the commission content input screen 1000generated in step S805 to the client terminal 101.

In step S807, the communication control unit 401 of the client terminal101 receives the commission content input screen 1000 transmitted fromthe medical image processing server 102. In step S808, the renderingunit 404 of the client terminal 101 renders the received commissioncontent input screen 1000, and the display unit 402 displays therendering results on the display 310 of the client terminal 101. Theuser, who is the commissioning party, then makes input of the commissioncontents to the commission content input screen 1000 that has beendisplayed in this way.

FIG. 11 is a flowchart diagram illustrating a series of processing foraccepting commissioning. The steps of step S1101, step S1102, and stepS1115 in FIG. 11 are processing executed by the CPU 301 of the clientterminal 101. The steps of step S1103 through step S1114 in FIG. 11 areprocessing executed by CPU 301 of the medical image processing server102. Note that the contents of processing and the order of processingillustrated in FIG. 11 are only exemplary, and are not restrictive.

In step S1101, determination is made by the operation accepting unit 405of the client terminal 101 regarding whether or not a pressing operationof the OK button 1004 of the commission content input screen 1000 hasbeen accepted. In a case where determination is made that a pressingoperation of the OK button 1004 has been accepted, the flow advances tostep S1002. In a case where determination is made that a pressingoperation of the OK button 1004 has not been accepted, the flow standsby.

In step S1102, the Web browser unit 403 of the client terminal 101acquires the image ID embedded in the OK button 1004 that has beenpressed, and the commission contents input to the commission contentinput screen 1000 (formation part, formation method, confirmationformat, etc.). The communication control unit 401 of the client terminal101 then transmits the acquired image ID hand commission content to themedical image processing server 102, to commission formation of themedical image indicated by this image ID (referred to as “object medicalimage” in FIG. 11).

In step S1103, the communication control unit 411 of the medical imageprocessing server 102 receives the commission content and image IDtransmitted from the client terminal 101. In step S1104, the storageunit 412 of the medical image processing server 102 creates a new record(commission information) in a commission information management table1200, and registers the received commission contents and image ID inthis record.

The commission information management table 1200 is a data table forstoring information relating to formation commissions. The commissioninformation management table 1200 is stored in the external memory 311of the medical image processing server 102. Note that the configurationof the commission information management table 1200 is only exemplary,and is not restrictive.

The commission information management table 1200 includes the items ofcommission ID 1201, image ID 1202, formation part 1203, formation method1204, formation necessity 1205, confirmation format 1206, andcommissioning party information 1207. The commission informationmanagement table 1200 further includes the items of status 1208, 3D datasaving location 1209, and 3D PDF data saving location 1210.

The commission ID 1201 is an item storing identification informationuniquely allocated to each commission information. The image ID 1202 isan item storing the image ID 601 of the medical image used to commissionforming. The formation part 1203 is an item storing the formation partaccepted by selection at the formation part selection space 1001. Theformation method 1204 is the formation method of which selection hasbeen accepted at the formation method section space 1002. The formationnecessity 1205 is an item storing the formation necessity of whichselection has been accepted at the formation method section space 1002.The confirmation format 1206 is an item storing the confirmation formatof which selection has been accepted at the conformation formatselection space 1003. The commissioning party information 1207 is anitem storing information of the commissioning party, such as name andaddress. The status 1208 is an item storing the state of progress of theformation commission. In later-described processing, when a commissionhas been accepted this is commission accepted, once finalization of thecommission is performed this is commission finalized, once the 3Dforming apparatus is instructed to perform the 3D forming, the status isforming, when formation is completed this is billing, and when thebilled payment has been received, the status is commission completed.Other statuses may be stored as well, as necessary. The 3D data savinglocation 1209 is an item storing the saving location of generated 3Ddata, which will be described later. The 3D PDF data saving location1210 is an item storing the saving location of generated 3D PDF data,which will be described later.

In step S1105, the volume data generating unit 418 of the medical imageprocessing server 102 acquires the object medical image from the medicalimage saving location 607, and generates volume data using the objectmedical image. There are multiple object medical images, and the volumedata is generated by layering these. Conventional art is used togenerate volume data, so detailed description will be omitted.

In step S1106, the formation part extracting unit 419 of the medicalimage processing server 102 extracts the formation part 1203 regardingwhich the commission has been accepted, from the volume data generatedin step S1105. The formation part extracting unit 419 identifies theformation part 904 corresponding to the formation part 1203, andautomatically extracts the commissioned human body part by using anextraction algorithm 905 corresponding to the formation part 904. Themethod for extracting a particular part from volume data also isconventional art, so detailed description will be omitted. Referencenumeral 1301 in FIG. 13 denotes an overview of generating volume data insteps S1105 and S1106, and extracting the specified part. Thus, a 3Dimage of a specified part is generated from multiple medical images.FIG. 13 illustrates an example of the liver.

In step S1107, the 3D control unit 417 of the medical image processingserver 102 determines whether or not extraction of the commissionedhuman body part has succeeded. There is a possibility that thecommissioned human body part cannot be extracted even by using theextraction algorithm 905, if the medical images are unclear or thenumber of medical images is insufficient. Accordingly, in a case wherethe extraction algorithm 905 is executed and an error is returned,determination is made that extraction has failed. In a case wheredetermination is made that extraction of the commissioned human bodypart has been successful, the flow advances to step S1109. Otherwise,i.e., in a case where determination is made that extraction has failed,the flow advances to step S1108.

In step S1108, the communication control unit 411 of the medical imageprocessing server 102 transmits processing results to the clientterminal 101, to the effect that the commissioned extraction of thehuman body part has failed. In step S1115, the communication controlunit 401 of the client terminal 101 accepts the processing results, andnotifies the user, who is the commissioning party.

On the other hand, in a case where extraction has been successful, instep S1109 the 3D data generating unit 420 of the medical imageprocessing server 102 generates 3D data of the part extracted in stepS1116. 3D data that can be analyzed by the 3D forming driver unit 415,such as VRML or STL format data is generated using the volume data fromwhich the commission part has been extracted. Generating 3D data alsouses conventional art, so description will be omitted. This processingcorresponds to the part indicated by 1302 in FIG. 13.

In step S1110, the storage unit 412 of the medical image processingserver 102 saves the 3D data generated in step S1109 in the externalmemory 311 of the medical image processing server 102. The location ofsaving this 3D data is stored in the 3D data saving location 1209 of thecorresponding record.

In step S1111, the storage unit 412 of the medical image processingserver 102 determines whether or not the confirmation format 1206 of therecord corresponding to the commission being processed is 3D PDF data.That is to say, determination is made regarding whether or not the user,who is the commissioning party, has selected from the conformationformat selection space 1003 to confirm in 3D PDF data. In a case wheredetermination is made that the confirmation format 1206 is 3D PDF data,the flow advances to S1112. In a case where determination is made thatthe confirmation format 1206 is not 3D PDF data, i.e., is 3D data, theflow advances to step S1114.

In step S1112, the 3D PDF data generating unit 421 of the medical imageprocessing server 102 generates 3D PDF data including the 3D datagenerated in step S1109. The 3D data is acquired from the 3D data savinglocation 1209 of the record corresponding to the commission beingprocessed, and PDF data including this 3D data is generated. The methodof generating 3D PDF data also uses conventional art, so descriptionwill be omitted. This processing corresponds to the portion in FIG. 13indicated by reference numeral 1303.

In step S1113, the storage unit 412 of the medical image processingserver 102 saves the 3D PDF data generated in step S1112 in the externalmemory 311 of the medical image processing server 102. The savinglocation is then stored in the 3D PDF data saving location 1210 of therecord corresponding to the commission being processed.

In step S1114, the communication control unit 411 of the medical imageprocessing server 102 transmits processing results to the clientterminal 101, to the effect that extraction of the commission part andgeneration of the 3D data or 3D PDF data has been successful. Thecommunication control unit 401 of the client terminal 101 receives theprocessing results in step S1115, and notifies the user, who is thecommissioning party.

Once the processing up to step S1115 is completed, the client terminal101 updates the medical image list screen 700. That is to say, theseries of processing illustrated in FIG. 5 is executed. Thus, themedical image list screen 700 is updated to the newest state.

FIG. 14 is a flowchart diagram illustrating a series of processing fordisplaying commission results. The steps of step S1401, step S1402, stepS1406, and step S1407 in FIG. 14 are processing executed by the CPU 301of the client terminal 101. The steps of step S1403 through step S1405in FIG. 14 are processing executed by CPU 301 of the medical imageprocessing server 102. Note that the contents of processing and theorder of processing illustrated in FIG. 14 are only exemplary, and arenot restrictive.

In step S1401, the operation accepting unit 405 of the client terminal101 determines whether or not an operation of pressing the confirmbutton 703 of the medical image list screen 700 has been accepted. In acase where determination is made that pressing of the confirm button 703has been accepted, the flow advances to step S1402. In a case wheredetermination is made that pressing of the confirm button 703 has notbeen accepted, the flow stands by.

In step S1402, the Web browser unit 403 of the client terminal 101acquires the image ID embedded in the record of the confirm button 703regarding which the pressing operation has been accepted. Thecommunication control unit 401 of the client terminal 101 then transmitsa commission results acquisition request for the medical image that thisimage ID indicates (referred to as “object medical image” in FIG. 14) tothe medical image processing server 102 along with this image IDincluded.

In step S1403, the communication control unit 411 of the medical imageprocessing server 102 receives the commission results acquisitionrequest transmitted from the client terminal 101.

In step S1404, the storage unit 412 of the medical image processingserver 102 acquires the processing results of the object medical imagefrom the external memory 311. The image ID of the object medical imagehas been received from the client terminal 101, so the image ID 1202corresponding to this image ID is identified from the records in thecommission information management table 1200. In a case where theconfirmation format 1206 of the identified record is 3D data, the 3Ddata, which is the commission results, is acquired from the 3D datasaving location 1209. On the other hand, in a case where theconfirmation format 1206 of the identified record is 3D PDF data, the 3DPDF data, which is the commission results, is acquired from the 3D PDFdata saving location 1210.

In step S1405, the communication control unit 411 of the medical imageprocessing server 102 transmits the processing results acquired in stepS1404 (3D data or 3D PDF data) to the client terminal 101.

In step S1406, the communication control unit 411 of the client terminal101 receives the processing results transmitted from the medical imageprocessing server 102. In step S1407, in a case where 3D data has beenreceived as the processing results, the client terminal 101 saves the 3Ddata in the external memory 311. The saved 3D data is then displayed inaccordance with user instructions, by an application installed in theclient terminal 101. On the other hand, in a case where 3D PDF data hasbeen received as the processing results, the display unit 402 of theclient terminal 101 start up a PDF viewer installed in the clientterminal 101, and displays the received 3D PDF data on the display 310.

FIG. 15 illustrates an example of displaying 3D PDF data using a PDFviewer. The operating the 3D data in this view enables the viewpoint tobe changed. Thus, the user can confirm the commissioning results beforecommissioning formation. Also, even in cases where the user, who is thecommissioning party, performs the formation him/herself, 3D data thatcan be output to the 3D forming apparatus 103 can be downloaded.

FIG. 16 is a flowchart diagram illustrating a series of processing forfinalizing commission. The steps of step S1601 and step S1602 in FIG. 16are processing executed by the CPU 301 of the client terminal 101. Thesteps of step S1603 through step S1609 in FIG. 16 are processingexecuted by CPU 301 of the medical image processing server 102. Notethat the contents of processing and the order of processing illustratedin FIG. 16 are only exemplary, and are not restrictive.

In step S1601, the operation accepting unit 405 of the client terminal101 determines whether or not an operation of pressing the finalizebutton 704 of the medical image list screen 700 has been accepted. In acase where determination is made that pressing of the finalize button704 has been accepted, the flow advances to step S1602. In a case wheredetermination is made that pressing of the finalize button 704 has notbeen accepted, the flow stands by.

In step S1602, the Web browser unit 403 of the client terminal 101acquires the image ID embedded in the record of the finalize button 704regarding which the pressing operation has been accepted. Thecommunication control unit 401 of the client terminal 101 then transmitsa final finalization request for the medical image that this image IDindicates (referred to as “object medical image” in FIG. 16) to themedical image processing server 102 along with this image ID included.

In step S1603, the communication control unit 411 of the medical imageprocessing server 102 receives the finalization request transmitted fromthe client terminal 101. In step S1604, the storage unit 412 of themedical image processing server 102 determines whether or not formationof the object medical image regarding which the finalization request hasbeen made is necessary. The image ID of the object medical image hasbeen received from the client terminal 101, so the image ID 1202corresponding to this image ID is identified from the records in thecommission information management table 1200. In a case where theformation necessity 1205 of the identified record indicates necessary,the flow advances to step S1605. Otherwise, i.e., in a case whereformation is not necessary, the flow advances to step S1608.

In step S1605, the 3D forming driver unit 415 of the medical imageprocessing server 102 acquires the 3D data of the object medical imagefrom the external memory 311 of the medical image processing server 102.This can be acquired from the 3D data saving location 1209 of the recordidentified in step S1604.

In step S1606, the formation job generating unit 416 of the medicalimage processing server 102 generates a formation job using the 3D dataacquired in step S1605. At this time, the formation job is generatedincluding the formation method 1204 and further any informationregarding the material for forming. Generating formation jobs isperformed using conventional art.

In step S1607, the 3D forming driver unit 415 of the medical imageprocessing server 102 transmits the formation job generated in stepS1606 to the 3D forming apparatus 103, thereby giving a formationinstruction. At this time, the 3D forming driver unit 415 identifies a3D forming apparatus 103 as the transmission destination that is capableof executing the formation method 1204 identified in step S1604, andtransmits to the 3D forming apparatus 103. Upon receiving the formationjob, the 3D forming apparatus 103 starts the 3D forming. When theforming is completed, the business delivers the formed object to theuser, who is the commissioning party, based on the commissioning partyinformation 1207.

In step S1608, the medical image processing server 102 issues an invoiceto the user, who is the commissioning party, in accordance with thereceived commission contents. In step S1609, the storage unit 412 of themedical image processing server 102 updates the status 1208 of therecord identified in step S1604 to billing. Thus the formationcommission is finalized.

Once the processing up to step S1609 is completed, an update request forthe medical image list screen 700 is transmitted from the medical imageprocessing server 102 to the client terminal 101. Upon receiving thisupdate request, the client terminal 101 executes the series ofprocessing illustrated in FIG. 5. Thus, the medical image list screen700 is updated to the newest state.

FIG. 17 is a flowchart diagram illustrating a series of processing forcancelling a commission. The steps of step S1701 and step S1702 in FIG.17 are processing executed by the CPU 301 of the client terminal 101.The steps of step S1703 through step S1705 in FIG. 17 are processingexecuted by CPU 301 of the medical image processing server 102. Notethat the contents of processing and the order of processing illustratedin FIG. 17 are only exemplary, and are not restrictive.

In step S1701, the operation accepting unit 405 of the client terminal101 determines whether or not an operation of pressing the cancel button705 of the medical image list screen 700 has been accepted. In a casewhere determination is made that pressing of the cancel button 705 hasbeen accepted, the flow advances to step S1702. In a case wheredetermination is made that pressing of the cancel button 705 has notbeen accepted, the flow stands by.

In step S1702, the Web browser unit 403 of the client terminal 101acquires the image ID embedded in the record of the cancel button 705regarding which the pressing operation has been accepted. Thecommunication control unit 401 of the client terminal 101 then transmitsa cancel request for the medical image that this image ID indicates(referred to as “object medical image” in FIG. 17) to the medical imageprocessing server 102 along with this image ID included.

In step S1703, the communication control unit 411 of the medical imageprocessing server 102 receives the cancel request transmitted from theclient terminal 101. In step S1704, the storage unit 412 of the medicalimage processing server 102 deletes the commission results of thecommission regarding which the cancel request has been made from theexternal memory 311. The image ID has been received from the clientterminal 101, so the image ID 1202 corresponding to this image ID isidentified from the records in the commission information managementtable 1200. The 3D data and the 3D PDF data, that are the commissionresults, are then deleted from the 3D data saving location 1209 and 3DPDF data saving location 1210 of the identified record.

In step S1705, the storage unit 412 of the medical image processingserver 102 deletes the commission information regarding which the cancelrequest has been made, i.e., the record identified in step S1704, fromthe commission information management table 1200. Thus, the formationcommission is canceled.

Once the processing up to step S1705 is completed, an update request forthe medical image list screen 700 is transmitted from the medical imageprocessing server 102 to the client terminal 101. Upon receiving thisupdate request, the client terminal 101 executes the series ofprocessing illustrated in FIG. 5. Thus, the medical image list screen700 is updated to the newest state.

As described above, redundant formation commissioning of medical imagescan be suppressed.

Another embodiment of the present invention will be described in detailwith reference to the drawings. First, an overview of the presentinvention will be described with reference to FIG. 18. A medical imageprocessing system 2100 according to the present invention is a systemincluding a client terminal 2101 and a medical image processing server2102. The client terminal 2101 and medical image processing server 2102are communicably connected.

The client terminal 2101 transmits medical images generated bymodalities such as X-ray CT or MRI to the medical image processingserver 2102 beforehand. The medical image processing server 2102 storesthese medical images. The term “medical image” in the present embodimentincludes tomographic images generated by one modality or another. Theterm “medical image” also includes images according to the DigitalImaging and COmmunication in Medicine (DICOM) standard. Accessoryinformation for DICOM includes information of examination date, patientname, type of examination, part or site name, and so forth. That is tosay, a medical information includes this information. When storingmedical images in the medical image processing server 2102, multipleimages acquired in a single examination are made into a series (grouped)and stored. Note that in the present embodiment, this medical imagegroup that has been made into a series will be referred to as “medicalimage”.

When generating text document data into which 3D data representing ahuman body part is inserted, i.e., PDF data (hereinafter referred to as3D PDF data), the client terminal 2101 accepts an instruction to thateffect. A screen such as indicated by reference numeral 2110 isdisplayed on a display of the client terminal 2101, and the user, who isthe commissioning party, presses a “commission” button provided to arecord of an object medical image. Upon detecting this pressing, ascreen such as indicated by reference numeral 2120 is displayed on adisplay 2120 of the client terminal 2101. Selection of the human bodypart to be displayed three-dimensional in 3D PDF data is accepted atthis screen.

At the time of generating this screen at the medical image processingserver 2102, parts regarding which 3D data can be generated aredetermined using the information of the parts included in the objectmedical image regarding which the commission button has been pressed.Checkboxes are provided so that parts that can be generated, can beselected. Thus, the user can be suppressed from selecting partsregarding which 3D data cannot be generated from the medical imagesregarding which selection has been accepted.

Once the user selects a part to three-dimensionally display, and theclient terminal 2101 detects pressing of an OK button, a 3D PDF datagenerating commission is transmitted from the client terminal 2101 tothe medical image processing server 2102. The medical image processingserver 2102 accepts this commission, and generates volume data using theobject medical image.

Volume data is data where medical images are layeredthree-dimensionally, and a CT number (a relative value where water is 0and air is −1000) that each pixel in the medical image has is given as aparameter to a voxel corresponding to that pixel. The medical imageprocessing server 2102 applies visual information (color, viewpoint,slight source, gloss, etc.) to the volume data, thereby generating 3Ddata. This 3D data is data representing a 3D model in the VRML format orSTL format, for example.

The 3D data generated in this way is inserted into PDF data, therebygenerating 3D PDF data. The user, who is the commissioning party, opensthis 3D PDF data using a PDF viewer installed in the client terminal2101, thus enabling three-dimensional display of the human body part.This overview will be described in further detail.

FIG. 19 is a diagram illustrating the system configuration of themedical image processing system 2100. Note that the system configurationillustrated in FIG. 19 is only exemplary. The medical image processingsystem 2100 is a system including the client terminal 2101 and medicalimage processing server 2102. The client terminal 2101 and medical imageprocessing server 2102 are communicably connected via a LAN 2201 or theInternet or the like.

The client terminal 2101 is one or multiple devices at the side of thecommissioning user that commissions 3D PDF data generation. The clientterminal 2101 may be a personal computer or server, or may be a mobileterminal (cellular phone, smartphone, wearable device, etc.) or thelike.

The medical image processing server 2102 (medical image processingapparatus) is one or multiple devices at the side of a business thataccepts commissions for 3D PDF data generation. Although a server deviceis assumed as the medical image processing server 2102, any devicecapable of processing commissions from the client terminal 2101 may beused.

FIG. 20 is a diagram that illustrates the hardware configuration of theclient terminal 2101 and medical image processing server 2102. Note thatthe hardware configuration in FIG. 20 is only exemplary. A centralprocessing unit (CPU) 2301 centrally controls various devices andcontrollers connected to a system bus 2304.

ROM 2302 or external memory 2311 store a BIOS and operating systemsoftware, which are control programs for the CPU 2301. The externalmemory 2311 also stores programs that are necessary for the devices torealize the functions that they execute. RAM 2303 functions as mainmemory, work area, and so forth, of the CPU 2301. The display device maybe a CRT, liquid crystal display, or the like.

The CPU 2301 loads programs and the like necessary to execute processingto the RAM 2303, and executes the programs, thereby realizing thevarious types of operations. An input controller 2305 controls inputfrom an input device 2309 such as a keyboard or a pointing device like amouse or the like. A video controller 2306 controls display on a displaydevice such as a display 2310 or the like.

A memory controller 2307 controls access to the external memory 2311.Examples of the external memory 2311 include a hard disk, flexible disk,card-type memory connected to a PCMCIA card slot via an adapter, and soforth. The external memory 2311 (storage unit) stores a boot program,browser software, various types of applications, font data, user files,editing files, various types of data, and so forth.

A communication interface controller 2308 is for connection to andcommunication with external devices via a network, and executescommunication control processing on the network. For example, Internetcommunication using TCP/IP can be performed.

The CPU 2301 also enables display to be made on the display 2310 byrendering (rasterizing) outline fonts to a display information area inthe RAM 2303. The CPU 2301 further also enables user instructions on thedisplay 2310 by a mouse cursor or the like, omitted from illustration.

The various types of programs and the like used for the client terminal2101 and medical image processing server 2102 to execute various typesof processing that will be described later, are recorded in the externalmemory 2311. These various types of programs and so forth are loaded tothe RAM 2303 as necessary, and thus are executed by the CPU 2301.Further, definition files various types of information tables used byprograms according to the present invention are stored in the externalmemory 2311.

FIG. 21 is a diagram illustrating an example of the functionalconfiguration of the client terminal 2101 and medical image processingserver 2102. The functions illustrated in FIG. 21 are the componentsrealized by the hardware arrangements, various types of programs, and soforth, illustrated in FIG. 19. Note that the functional configurationillustrated in FIG. 21 is only exemplary. The client terminal 2101includes a communication control unit 2401, a display unit 2402, a Webbrowser unit 2403, and a PDF viewer unit 2407, as functional units.

The communication control unit 2401 is a functional unit for exchangingvarious types of information with the medical image processing server2102. The communication control unit 2401 communicates with the medicalimage processing server 2102 via the communication interface controller2308 of the client terminal 2101.

The display unit 2402 is a functional unit for displaying various typesof information. The display unit 2402 transmits images including varioustypes of information to the display 2310 via the video controller 2306of the client terminal 2101, so as to display the various types ofinformation on the display 2310.

The Web browser unit 2403 is a functional unit of a Web browserinstalled in the client terminal 2101 for displaying and operating Webpages (referred to as “screens” in the present embodiment), described inHTML. The Web browser unit 2403 further includes a rendering unit 2404,an operation accepting unit 2405, and a 3D data plugin unit 2406.

The rendering unit 2404 is a functional unit that analyzes the contentsdescribed in the Web page, and displays characters and images on thedisplay. The operation accepting unit 2405 is a functional unit thataccepts operations made regarding the Web page. The operation acceptingunit 2405 also transmits acquisition requests for Web pages and files tothe medical image processing server 2102 via the communication controlunit 2401, in accordance with accepted operations. The 3D data pluginunit 2406 is a functional unit that displays 3D data inserted into a Webpage. The 3D data plugin unit 2406 operates as a plugin of the Webbrowser, and analyzes content indicated by the 3D data to be rendered atthe rendering unit 2404. The 3D data plugin unit 2406 further operatesto change the display position, display orientation, etc., of the 3Ddata, in accordance with operations accepted at the operation acceptingunit 2405.

The PDF viewer unit 2407 is a PDF viewer function unit installed in theclient terminal 2101, and is a functional unit that displays PDF dataincluding 3D PDF data. The PDF viewer unit 2407 further includes a 3Ddata control unit 2408. The 3D data control unit 2408 is a function unitthat displays 3D data included in 3D PDF data. The 3D data control unit2408 operates to analyze and render the content that the 3D dataindicates, and to change the display position, display orientation,etc., of the 3D data, in accordance with operations accepted.

The medical image processing server 2102 includes, as functional units,a communication control unit 2411, a storage unit 2412, a screengenerating unit 2413, and a 3D control unit 2417.

The communication control unit 2411 is a functional unit that exchangesvarious types of information with the client terminal 2101. Thecommunication control unit 2411 communicates with the client terminal2101 on the network via the communication interface controller 2308 ofthe medical image processing server 2102.

The storage unit 2412 is a functional unit that stores various types ofinformation in the RAM 2303 or external memory 2311. Examples of thevarious types of information that the storage unit 2412 stores includesvarious types of tables that will be described later, templates for Webpages to be displayed at the client terminal 2101, and so forth.

The screen generating unit 2413 is a functional unit that generatesscreens (Web pages) to be displayed on the client terminal 2101. Thescreen generating unit 2413 generates screens using the various types ofinformation stored in the storage unit 2412. The screen generating unit2413 further includes a display part determining unit 2414, a 3D datainserting unit 2415, and a capture image insertion unit 2416.

The display part identifying unit 2414 is functional unit that acquiresmedical images instructed by the client terminal 2101 from the storageunit 2412, and identifies human body parts regarding which 3D data canbe generated from the medical images. The display part identifying unit2414 identifies human body parts regarding which 3D data can begenerated, from information of the examined parts included in the DICOMinformation of the medical images.

The 3D data inserting unit 2415 is a functional unit that inserts 3Ddata into the generated Web pages. 3D data may be embedded into a Webpage, or may be linked to from the Web page. The capture image insertionunit 2416 is a functional unit that changes the display orientation ofthe 3D data to a display orientation instructed by the user, capturesthis and generates image data (hereinafter referred to as a captureimage), and embeds this in a Web page. The capture image also may beembedded into a Web page, or may be linked to from the Web page.

The 3D control unit 2417 is a functional unit for performing controlsuch as generating and displaying three-dimensional data inthree-dimensional space. The 3D control unit 2417 further includes avolume data generating unit 2418, a formation part extracting unit 2419,the 3D data generating unit 2420, a 3D PDF data generating unit 2421,and a display orientation changing unit 2422.

The volume data generating unit 2418 is a functional unit that generatesvolume data from medical images. The volume data generating unit 2418generates volume data using multiple medical images. In a case wheremedical images to generate volume data are insufficient, the volume datagenerating unit 2418 interpolates for insufficient portions. The methodof generating volume data is conventional art, and will be describedlater.

The display part extracting unit 2419 is a functional unit that extractsthe human body part selected at the client terminal 2101 from the volumedata. The storage unit 2412 stores a known algorithm for extracting eachhuman body part, and the display part extracting unit 2419 selects thenecessary algorithm from the storage unit 2412 to perform extracting.The method of extracting particular body parts from volume data, and thealgorithms thereof, also are conventional art, and will be describedlater.

The 3D data generating unit 2420 is a functional unit that generates 3Ddata using the volume data from which the formation part extracting unit2419 has extracted a particular body part. The 3D data generating unit2420 generates 3D data in formats such as the VRML and STL formats, inthe present embodiment. Although file formats capable of generating 3DPDF data such as VRML format 3D data are exemplified in the presentembodiment, this is not restrictive.

The 3D PDF data generating unit 2421 is a functional unit that generatesPDF data including 3D data (hereinafter referred to as 3D PDF data). The3D PDF data is PDF data where the user can view 3D data from anyviewpoint. The method of generating the 3D PDF data also is conventionalart, and will be described later.

The display orientation changing unit 2422 is a functional unit thatchanges the display orientation of the 3D data generated at the 3D datagenerating unit 2420. The display orientation changing unit 2422 changesthe display orientation of the 3D data to that optionally instructed bythe user, in order to generate the capture image at the capture imageinsertion unit 2416. When changing the display orientation, theviewpoint in the three-dimensional space in which the 3D data issituated may be changed, or the 3D data may be rotated.

FIG. 22 is a flowchart diagram illustrating a series of processing fordisplaying the medical image list screen 2700. The steps of step S2501,S2510, and step S2511 in FIG. 22 are processing executed by the CPU 301of the client terminal 2101. The steps of step S2502 through step S2509in FIG. 22 are processing executed by CPU 2301 of the medical imageprocessing server 2102. Note that the contents of processing and theorder of processing illustrated in FIG. 22 are only exemplary, and arenot restrictive.

In step S2501, the communication control unit 2411 of the clientterminal 2101 transmits an acquisition request for a medical image listscreen to the medical image processing server 2102.

In step S2502, the communication control unit 2411 of the medical imageprocessing server 2102 receives the acquisition request for the medicalimage list screen transmitted from the client terminal 2101. In stepS2503, the screen generating unit 2413 of the medical image processingserver 2102 acquires a template for a medical image list screen from thestorage unit 2412 and information stored in a medical image managementtable 2600 illustrated in FIG. 23, and generates a medical image listscreen 2700 such as illustrated in FIG. 24. The medical image listscreen 2700 includes a medical image list 2701, with a record beingcreated in the medical image list 2701 for each record in the medicalimage management table 2600. A corresponding image ID 2601 is embeddedin each record in the medical image list 2701.

When creating the screen in step S2503, none of a commission button2702, confirm button 2703, finalize button 2704 and cancel button 2705,which will are laid out in later-described processing, are disposed yet.Further, the medical image list 2701 includes a display button 2706 foreach medical image record. The display button 2706 is for displaying themedical image. Accordingly, the display button 2706 is disposed as abutton to acquire the medical image corresponding to the record wherethe display button 2706 is disposed from a medical image saving location2607, and displaying the medical image.

The medical image management table 2600 is a table for storinginformation relating to medical images stored in the external memory2311 of the medical image processing server 2102. The medical imagemanagement table 2600 is stored in the external memory 2311 of themedical image processing server 2102. Upon receiving a medical imagefrom the client terminal 2101 or an apparatus of one modality oranother, a new record is created in the medical image management table2600, various types of information are acquired from the medical imageDICOM tag, and stored in the medical image management table 2600. Notethat the table configuration of the medical image management table 2600illustrated here is only exemplary, and is not restrictive.

Items of the medical image management table 2600 are image ID 2601,examination date 2602, patient name 2603, examination type 2604, site2605, number 2606, and medical image saving location 2607. The image ID2601 is an item storing identification information that is uniquelyallocated to each medical image that has been received. The examinationdate 2602 is an item storing the date on which the medical image wasimaged by an apparatus of one modality or another. The patient name 2603is an item storing the full name of the patient. The site 2605 is anitem storing the site of imaging the medical image. The number 2606 isan item storing the number of received medical images that are stored.The medical image saving location 2607 is an item sorting informationindicating a folder where the medical images are stored.

In step S2504, the screen generating unit 2413 of the medical imageprocessing server 2102 references one of the records in the medicalimage management table 2600. Upon having completed the steps of stepS2505 through step S2508 regarding the referenced record, determinationis made regarding whether or not the processing of step S2505 throughstep S2508 has been completed for all records in the medical imagemanagement table 2600. If determination is made that the processing hasbeen completed, the flow advances to step S2509. That is to say, stepS2505 through step S2508 are looped through for all records in themedical image management table 2600. The individual steps in step S2505through step S2508 will be described next.

In step S2505, the screen generating unit 2413 of the medical imageprocessing server 2102 determines whether the number of medical imagesof the record being referenced in the number 2606 is a predeterminednumber or more, and also that commission information of the medicalimages does not exist in a commission information management table 21500illustrated in FIG. 32. If both are true, the flow advances to stepS2506, and otherwise, the flow advances to step S2507. Determination mayalso be made regarding whether the number of medical images is largerthan the predetermined number, and also that no commission informationexists. The number of medical images is set beforehand, to a number fromwhich volume data indicating body parts can be generated from medicalimages. Alternatively, the number may be based on a number that willkeep coarseness in the generated volume data to an inconspicuous level,or a number where the output 3D data will have smooth curved surfaces.

The commission information management table 21500 illustrated in FIG. 32is a data table storing information relating to generating commissions.One record of the commission information management table 21500 isreferred to as “commission information” in the present embodiment.Details of the commission information management table 21500 will bedescribed later.

In step S2506, the screen generating unit 2413 of the medical imageprocessing server 2102 disposes the commission button 2702 to the recordof the medical image list 2701 corresponding to the record beingreferenced. The commission button 2702 is a button that can accept agenerating commission (generating instructions) of 3D PDF data including3D data representing human body parts from a medical image, and generate3D PDF data including the 3D data. A medical image that has a sufficientnumber is a medical image regarding which a generating commission can bemade, so the commission button 2702 is disposed thereat. Further, in acase where a generating commission can be made but a commission hasalready been made, a redundant commission raises the processing load onthe medical image processing server 2102.

Accordingly, in a case where a generating commission can be made but acommission has already been made, the commission button 2702 is notprovided, while the commission button 2702 is provided if no generatingcommission has been made yet. By distinguishing this display from onemedical image to another, the processing load on the medical imageprocessing server 2102 can be reduced, and the user can recognize eachmedical image regarding which a commission has already been made.Moreover, as a result of the determination in step S2505, in a casewhere the number of medical images is less than the predetermined number(or may be equal to or less), step S2506 is not executed, so nocommission button 2702 is provided for the record of this medical image.Accordingly, the user can be prevented from making commissions ofmedical images with an insufficient number.

Although an arrangement has been described where the commission buttonis not disposed, so that further instructions to generate 3D PDF dataare not given for medical image records that have already beencommissioned, an arrangement may be made where the commission button isdisplayed in a way that the button cannot be pressed, therebyfacilitating user comprehension that the commissioning has already beenperformed. Alternatively, an arrangement may be made where thecommission button is displayed in a different color or shape, or with adifferent text string, thereby facilitating user comprehension that thecommissioning has already been performed. Further, an arrangement may bemade where the commission button is disposed, and the disposedcommission button can be pressed, but an error is returned when thecommission button is pressed and no commission is placed.

In step S2507, the screen generating unit 2413 of the medical imageprocessing server 2102 determines whether or not commission informationcorresponding to the medical image of the record being referenced existsin the commission information management table 21500, and whether or nota status 21507 of the commission information is “currently confirmingcommissioning party”. That is to say, determination is made regardingwhether or not generating of 3D PDF data using the medical image of therecord being reference has already been commissioned, and commissionresults corresponding to the commission have already been output. If so,the flow advances to step S2508, and otherwise, the flow is returned tostep S2504 and whether or not the loop has been completed is determined.In a case where determination is made that the loop has been completed,the flow advances to step S2509.

In step S2508, the screen generating unit 2413 of the medical imageprocessing server 2102 disposes the confirm button 2703, finalize button2704, and cancel button 2705 at the record of the commission button 2702corresponding to the record being referenced in the medical image list2701.

The confirm button 2703 is a button for confirming the commissionresults. When the confirm button 2703 is pressed, a screen where thegenerated 3D data can be viewed is displayed. The finalize button 2704is a button for finalizing the commission. When the finalize button 2704is pressed, the 3D PDF data is downloaded to the client terminal 2101,and billing processing according to the commission is performed. Thecancel button 2705 is a button for canceling the commission. The confirmbutton 2703 is thus displayed in a case where commission results areoutput, so the user who is the commissioning part can recognize for eachmedical image whether or not commission results have been output.

When step S2508 is complete, the flow returns to step S2504 anddetermination is made regarding whether or not the loop has beencompleted. If determination is made regarding that the loop has beencompleted, the flow advances to step S2509.

In step S2509, The communication control unit 2411 of the medical imageprocessing server 2102 transmits the medical image list screen 2700generated in step S2503 through step S2508 to the client terminal 2101.

In step S2510, the communication control unit 2401 of the clientterminal 2101 receives the medical image list screen 2700 transmittedfrom the medical image processing server 2102. In step S2511, therendering unit 2404 of the client terminal 2101 renders the receivedmedical image list screen 2700, and the display unit 2402 displays therendering results on the display 2310 of the client terminal 2101.

After displaying the medical image list screen 2700, the client terminal2101 executes the flowcharts illustrated in FIGS. 25, 29, 31, 35, 38,and 40, by parallel processing.

FIG. 25 is a flowchart diagram illustrating a series of processing fordisplaying a commissioning content input screen 21100. The steps of stepS2801, step S2801, step S2807, and step S2808 in FIG. 25 are processingexecuted by the CPU 2301 of the client terminal 2101. The steps of stepS2803 through step S2806 in FIG. 25 are processing executed by CPU 2301of the medical image processing server 2102. Note that the contents ofprocessing and the order of processing illustrated in FIG. 25 are onlyexemplary, and are not restrictive.

In step S2801, the operation accepting unit 2405 of the client terminal2101 determines whether or not an operation of pressing the commissionbutton 2702 in the medical image list screen 2700 has been accepted. Ina case where determination is made that an operation of pressing thecommission button 2702 has been accepted, the flow advances to stepS2802. In a case where determination is made that an operation ofpressing the commission button 2702 has not been accepted, the flowstands by.

In step S2802, the Web browser unit 2403 of the client terminal 2101acquires the image ID embedded in the record of the commission button2702 that has accepted a pressing operation. The communication controlunit 2401 of the client terminal 2101 then transmits an acquisitionrequest for a commission content input screen, for commissioninggeneration of 3D PDF data using of the medical image that this image IDindicates (“object medical image”) in FIG. 25.

In step S2803, the communication control unit 2411 of the medical imageprocessing server 2102 receives the acquisition request for thecommission content input screen transmitted from the client terminal2101.

In step S2804, the display part identifying unit 2414 of the medicalimage processing server 2102 identifies the display part from the partof the object medical image. That is to say, parts regarding which 3Ddata can be generated from the object medical image are identified. Thiswill be described in detail. The site 2605 and examination type 2604 ofthe object medical image are referenced, and identifies records having asite 2901 and examination type 2902 matching these from a display partmanagement table 2900 illustrated in FIG. 26. The display partmanagement table 2900 is a data table stipulating display parts 2904regarding which 3D data can be generated for each site 2901. Althoughdescription is made in the present embodiment that parts that can bedisplayed from medical images are identified using such a display partmanagement table 2900, parts regarding which 3D data can be generatedmay be identified by analyzing medical images instead. That is to say,medical images may be analyzed by image processing, and analysis maderegarding what parts are in the image. A method may also be used wherethe medical image is formed into volume data and three-dimensionallyanalyzed.

The display part management table 2900 is a data table for storinginformation relating to human body parts. The display part managementtable 2900 is stored in the external memory 2311 of the medical imageprocessing server 2102. Note that the configuration of the display partmanagement table 2900 is exemplary, and not restrictive.

The display part management table 2900 contains the items of site 2901,examination type 2902, type of organ 2903, display part 2904, extractionalgorithm 2905, sample data 2906, and display position 2907. The site2901 is an image storing information indicating the range of the humanbody in which the display part 2904 is included. The examination type2902 is an item storing information indicating the modality ofexamination type, such as X-ray CT, MRI, or the like. The type of organ2903 is an item storing information that indicating the type of organ towhich the display part 2904 belongs. The display position of the displaypart 2904 in a later-described display part selection space 21101 isdecided in accordance with the type of organ 2903. For example, eachclassification indicated by the type of organ 2903 is displayed on theformation part 2904. The display part 2904 is an item storinginformation indicating parts regarding which 3D data can be generated.The extraction algorithm 2905 is an item storing information indicatinga known algorithm for extracting the display part 2904 from the medicalimage. The sample data 2906 is an item sorting information indicatingthe saving location of the file of 3D data for display on alater-described default display orientation selection space 1103. The 3Ddata is 3D data indicating a sample of the display part 2904(hereinafter referred to as sample data). The display position 2907 isan item storing information indicating coordinates in three-dimensionalspace where the 3D data that the sample data 2906 represents is placed.This display part management table 2900 is stored in the external memory2311 of the medical image processing server 2102 beforehand, by the userwho manages the medical image processing server 2102.

In step S2805, the screen generating unit 2413 of the medical imageprocessing server 2102 generates a commission content input screen thatcan accept selection regarding parts that can be generated, identifiedin step S2804.

FIG. 28 illustrates an example of the commission content input screen21100. The commission content input screen 21100 has a display partselection space 21101, a display orientation selection space 21102, thedefault display orientation selection space 21103, and an OK button21104. The display part selection space 21101 is a checkbox styleselection space for the user, who is the commissioning party, to selectthe human body part to be three-dimensionally displayed. This selectionspace accepts selection of body parts identified in step S2804. That isto say, selection of parts not identified in step S2804 is not accepted.A selection space that only includes parts identified in step S2804 ispreferable. This arrangement enables selection of parts regarding which3D data can be generated, identified from the object medical image, tobe accepted, so even if the user, who is the commissioning party, doesnot have extensive knowledge of medical images, commissioning generatingof parts than cannot be generated can be suppressed.

Although description has been made that commissioning generating ofparts regarding which 3D data cannot be generated is suppressed by notproviding any selection space for parts not identified in S2804, anarrangement may be made where spaces for these parts are provided, butdisplayed as selection spaces that cannot be selected. Alternatively,these selection spaces may be displayed in different colors or shapes,or with the text string struck out. Further, an arrangement may be madewhere these items can be selected, but the OK button 21104 cannot bepressed. For example, the OK button 21104 may be hidden, or the color orshape changed. Further, control may be effected so that a generatingcommission is not performed by returning an error in a case where the OKbutton 21104 is pressed.

The display orientation selection space 21102 is a checkbox typeselection space for selecting the display orientation, in order toaccept changing instructions for the display orientation of the 3D datain the 3D PDF data. Upon accepting selection of a display orientationfrom the display orientation selection space 21102, an object can changethe display orientation in the 3D PDF data (a later-described captureimage) is disposed. That is, upon detection of this object beingpressed, the 3D data is rotated and the display orientationcorresponding to the object is displayed toward the user (at the nearside). The display orientation selection space 21102 displays a displayorientation name 1001 in a display orientation management table 21000illustrated in FIG. 27, as a selection item.

The display orientation management table 21000 is a data table sortinginformation relating to the display orientations selectable from thedisplay orientation selection space 21102. The display orientationmanagement table 21000 is stored in the external memory 2311 of themedical image processing server 2102. Note that the table configurationof the display orientation management table 21000 is only exemplary, andis not restrictive.

The display orientation management table 21000 includes as items, thedisplay orientation name 21001, rotation axis 21002, and rotation angle21003. The display orientation name 21001 is an item storing informationrepresenting the name of the display orientation. Six displayorientations are defined in FIG. 27 in the present embodiment, but thisis not restrictive. The rotation axis 21002 is an item storinginformation representing the rotation axis used to rotate to the displayorientation that the display orientation name 21001 indicates. Therotation angle 21003 is an item storing information representing therotation angle used to rotate to the display orientation that thedisplay orientation name 21001 indicates. The rotation axis 21002 androtation angle 21003 illustrated in FIG. 27 are stored as examples ofparameters assuming that the viewpoint in the three-dimensional space inwhich the 3D data is situated is not changed, and the 3D data is rotatedin the present embodiment. In an arrangement where the 3D data itself isnot rotated, and the viewpoint is changed in the three-dimensional spacein which the 3D data is situated, corresponding parameters are stored inthe rotation axis 21002 and rotation angle 21003.

Returning to FIG. 28, the default display orientation selection space21103 is a space for selection of a default display orientation of the3D data in the 3D PDF data (initial state when the 3D PDF data isopened). The default display orientation can be changed by the sampledata of the part selected by the display part selection space 21101being displayed in the default display orientation selection space21103, and the user changing the display orientation of the sample databy a dragging operation by a mouse or the like. The OK button 21104 is abutton for transmitting the commission contents accepted at thecommission content input screen 21100 to the medical image processingserver 2102. The image ID 601 of the object medical image is embedded inthe OK button 21104. Although omitted from illustration in thecommission content input screen 21100 in FIG. 27, an input form forinputting commissioning party information of the user, such as name,contact information (telephone number, address), and so forth, is alsoprovided.

In step S2806, the communication control unit 2411 of the medical imageprocessing server 2102 transmits the commission content input screen21000 generated in step S2805 to the client terminal 2101.

In step S2807, the communication control unit 2401 of the clientterminal 2101 receives the commission content input screen 21000transmitted from the medical image processing server 2102. In stepS2808, the rendering unit 2404 of the client terminal 2101 renders thereceived commission content input screen 21100, and the display unit2402 displays the rendering results on the display 2310 of the clientterminal 2101. The user, who is the commissioning party, then makesinput of the commission contents to the commission content input screen21100 that has been displayed in this way.

FIG. 29 is a flowchart diagram illustrating a series of processingexecuted in a case where selection of a display part has been accepted.The steps of step S21201, step S21202, and step S21210 in FIG. 29 areprocessing executed by the CPU 2301 of the client terminal 2101. Thesteps of step S21203 through step S21209 in FIG. 29 are processingexecuted by CPU 2301 of the medical image processing server 2102. Notethat the contents of processing and the order of processing illustratedin FIG. 29 are only exemplary, and are not restrictive.

In step S21201, the operation accepting unit 2405 of the client terminal2101 determines whether or not a selection has been accepted regardingthe display part displayed at the display part selection space 21101 onthe commission content input screen 21100. That is to say, determinationis made regarding whether or not any of the checkboxes in the displaypart selection space 21101 have been checked. In a case wheredetermination is made that a selection has been accepted regarding thedisplay part, the flow advances to step S21202. In a case wheredetermination is made that a selection has not been accepted regardingthe display part, the flow stands by.

In step S21202, the communication control unit 2401 of the clientterminal 2101 transmits an update request of the commission contentinput screen 21100, including information relating to the selecteddisplay part, to the medical image processing server 2102. In a casewhere selection has already been made regarding the display orientationat the default display orientation selection space 21103, informationrelating to the display orientation (rotation axis, rotation angle) alsois acquired, and included in the update request.

In step S21203, the communication control unit 2401 of the medical imageprocessing server 2102 receives the update request for the commissioncontent input screen 21100 transmitted from the client terminal 2101.

In step S21204, the 3D data generating unit 2420 of the medical imageprocessing server 2102 acquires sample data of the selected display partfrom information relating to the selected display part included in theupdate request received in step S21203. The selected display part isidentified from the display part 2904 in the display part managementtable 2900, and sample data is acquired from the saving location thatthe sample data 2906 of the identified record indicates.

In step S21205, the 3D data generating unit 2420 of the medical imageprocessing server 2102 places the sample data acquired in step S21204 atthe position in the three-dimensional space indicated by the displayposition 2907. The display position 2907 of the record identified instep S21204 is reference and the position of placement is decided. Thelayout relationship of each sample data here is preferably in the samelayout relationship as in the human body structure. That is to say, thesample data for each part is laid out in the same way as the structureof the human body. For example, if “lumbar spine” and “sacrum” are to belaid out, the “sacrum” is paced beneath the “lumbar spine”, as in ahuman body. Thus, if multiple display parts are selected, the layoutrelationship between these parts is the same layout relationship as in ahuman body, so natural 3D data of the structure of the human body can beoutput. In order to realize this, the layout relationship of thethree-dimensional space indicated by the display position 2907preferably stores positions taking this sort of layout relationship intoconsideration.

In step S21206, the 3D data generating unit 2420 of the medical imageprocessing server 2102 determines whether or not all sample datacorresponding to the selected display parts has been placed in thethree-dimensional space. In a case where determination is made that alldata has been placed, the flow advances to step S21207. Otherwise, theflow returns to step S21204. Thus, all selected display parts are placedin the three-dimensional space.

In step S21207, the 3D data generating unit 2420 of the medical imageprocessing server 2102 generates 3D data including all sample dataplaced in the three-dimensional space. That is to say, in a case where“lumbar spine” and “pelvis” have been selected from the display partselection space 21101, sample data of these two is placed in thethree-dimensional space. 3D data made up of the sample data of these twois then generated.

In step S21208, the 3D data inserting unit 2415 of the medical imageprocessing server 2102 inserts the 3D data generated in step S21207 tothe default display orientation selection space 21103 of the commissioncontent input screen 21100 generated in step S2805. The term insert maymean embedding 3D data into the screen, and may mean the 3D data beinglinked to from the screen. The 3D data displayed in the default displayorientation selection space 21103 thus changes in accordance with thedisplay parts selected in the display part selection space 21101.Accordingly, the user can change the default display orientation whileoperating 3D data in a state near to that of the commission results.

In step S21209, the communication control unit 2411 of the medical imageprocessing server 2102 transmits the commission content input screen21100 into which the 3D data has been inserted in step S21208 to theclient terminal 2101.

In step S21210, the communication control unit 2401 of the clientterminal 2101 receives the commission content input screen 21100transmitted from the medical image processing server 2102. The renderingunit 2404 then renders the received commission content input screen21100, and the display unit 2402 displays the rendered results on thedisplay 2310 of the client terminal 2101. Although the presentembodiment has been described with the entire screen (Web page) beingupdated, an arrangement may be made where only partial updating is madeof the default display orientation selection space 21103.

FIG. 30 illustrates an example of the results of having updated thecommission content input screen 21100 in this way. FIG. 30 is thecommission content input screen 21100 in a case of “liver” having beenselected at the display part selection space 21101. AS a result of“liver” having been selected, a record where the display part 2904 inthe display part management table 2900 is “liver” is identified, andsample data for “liver” is acquired from the sample data 2906 of theidentified record. This sample data is then placed at the position inthe three-dimensional space indicated by the display position 2907 ofthe identified record, and 3D data is generated. Inserting the generated3D data into the default display orientation selection space 21103 ofthe commission content input screen 21100 yields what is illustrated inFIG. 30. The 3D data displayed on the default display orientationselection space 21103 can be rotated on a center point of the 3D data bya dragging operation by an input device such as a mouse or the like.This rotation enables the display orientation to be freely set.

FIG. 31 is a flowchart diagram illustrating a series of processing ofaccepting commission content. The steps of step S21401, step S21402, andstep S21419 in FIG. 31 are processing executed by the CPU 2301 of theclient terminal 2101. The steps of step S21403 through step S21418 inFIG. 31 are processing executed by CPU 2301 of the medical imageprocessing server 2102. Note that the contents of processing and theorder of processing illustrated in FIG. 31 are only exemplary, and arenot restrictive.

In step S21401, the operation accepting unit 2405 of the client terminal2101 determines whether or not a pressing of the OK button 21104 of thecommission content input screen 21100 has been accepted. In a case wheredetermination is made that a pressing of the OK button 21104 has beenaccepted, the flow advances to step S21402. In a case wheredetermination is made that a pressing of the OK button 21104 has notbeen accepted, the flow stands by.

In step S21402, the Web browser unit 2403 of the client terminal 2101acquires the image ID embedded in the OK button 21104 that has acceptedthe pressing operation, and the commission contents (display part,display orientation, default display orientation, etc.) input to thecommission content input screen 21100. The communication control unit2401 of the client terminal 2101 generates 3D data of the selecteddisplay part from the medical image indicated by this image ID (referredto as “object medical image” in FIG. 31), and commissions the medicalimage processing server 2102 to generate 3D PDF data. That is to say,the acquired image ID and commission contents are transmitted to themedical image processing server 2102.

In step S21403, the communication control unit 2411 of the medical imageprocessing server 2102 receives the commission contents and image IDtransmitted from the client terminal 2101 (accepting unit). That is tosay, the contents selected at the commission content input screen 21100are accepted. In step S21404, the storage unit 2412 of the medical imageprocessing server 2102 creates a new record (commission information) ina commission information management table 21500 illustrated in FIG. 32,and registers the received commission contents and image ID in thisrecord.

The commission information management table 21500 is a data table thatstores information relating to generating commissions of 3D PDF data.The commission information management table 21500 is stored in theexternal memory 2311 of the medical image processing server 2102. Notethat the table configuration of the commission information managementtable 21500 is only exemplary, and is not restrictive.

Items that the commission information management table 21500 has are acommission ID 21501, image ID 21502, display part 21503, displayorientation 21504, default 21505, and commissioning party information21506. Further included are status 21507, 3D data saving location 21508,and 3D PDF data saving location 21509.

The commission ID 21501 is an item storing identifying informationuniquely allocated to each commission information. The image ID 21502 isan item storing the image ID 2601 of the medical image used forgenerating 3D data that has been commissioned. The display part 21503 isan item storing the display part that has been selected at the displaypart selection space 21101. The display orientation 21504 is an itemstoring the display orientation that has been selected in the displayorientation selection space 21102. The default 21505 is an item storingthe default display orientation that has been selected in the defaultdisplay orientation selection space 21103. The commissioning partyinformation 21506 is an item storing information of the commissioningpart such as name and address. The status 21507 is an item storing thestate of progress of the commission for generating 3D PDF data. When acommission has been accepted, this is commission accepted, when thecommission results are output in later-described processing, this isconfirming at commissioning party, once finalization of the commissionis performed this is commission finalized, when the commission contentsare completed this is billing, and when the billed payment has beenreceived, the status is commission completed. Other statuses may bestored as well, as necessary. The 3D data saving location 21508 is anitem storing the saving location of generated 3D data, which will bedescribed later. The 3D PDF data saving location 21509 is an itemstoring the saving location of generated 3D PDF data, which will bedescribed later.

In step S21405, the volume data generating unit 2418 of the medicalimage processing server 2102 acquires the object medical image from themedical image saving location 2607, and generates volume data using theobject medical image. There are multiple object medical images, and thevolume data is generated by layering these. Conventional art is used togenerate volume data, so detailed description will be omitted.

In step S21406, the display part extracting unit 2419 of the medicalimage processing server 2102 extracts the display part 21503 regardingwhich the commission has been accepted, from the volume data generatedin step S21405. The display part extracting unit 2419 identifies thedisplay part 2904 corresponding to the display part 21503, andautomatically extracts the commissioned human body part by using anextraction algorithm 2905 corresponding to the display part 2904. Themethod for extracting a particular part from volume data also isconventional art, so detailed description will be omitted. Referencenumeral 21601 in FIG. 33 denotes an overview of generating volume datain steps S21405 and S21406, and extracting the specified part. Thus, a3D image of a specified part is generated from multiple medical images.

FIG. 33 illustrates an example of the liver. In step S21407, the 3Dcontrol unit 2417 of the medical image processing server 2102 determineswhether or not extraction of the commissioned human body part hassucceeded. There is a possibility that the commissioned human body partcannot be extracted even by using the extraction algorithm 2905, if themedical images are unclear or the number of medical images isinsufficient. Accordingly, in a case where the extraction algorithm 2905is executed and an error is returned, determination is made thatextraction has failed. In a case where determination is made thatextraction of the commissioned human body part has been successful, theflow advances to step S21409. Otherwise, i.e., in a case wheredetermination is made that extraction has failed, the flow advances tostep S21408.

In step S21408, the communication control unit 2411 of the medical imageprocessing server 2102 transmits processing results to the clientterminal 2101, to the effect that the commissioned extraction of thehuman body part has failed. In step S21419, the communication controlunit 2401 of the client terminal 2101 accepts the processing results,and notifies the user, who is the commissioning party.

On the other hand, in a case where extraction has been successful, instep S21409 the 3D data generating unit 2420 of the medical imageprocessing server 2102 generates 3D data of the part extracted in stepS21406 (three-dimensional data generating unit). VRML or STL format datais generated using the volume data from which the commission part hasbeen extracted. Generating 3D data also uses conventional art, sodescription will be omitted. This processing corresponds to the partindicated by 21602 in FIG. 33.

In step S21410, the storage unit 2412 of the medical image processingserver 2102 saves the 3D data generated in step S21409 in the externalmemory 2311 of the medical image processing server 2102. The location ofsaving this 3D data of the commission information being processed, thathas been newly created in step S21404, is stored in the 3D data savinglocation 21508 of the corresponding record.

In step S21411, the 3D PDF data generating unit 2421 of the medicalimage processing server 2102 generates 3D PDF data including the 3D datagenerated in display object 1409 (document data generating unit) in stepS21409. The 3D data is acquired from the 3D data saving location 21508of the commission information being processed, and PDF data (documentdata) including this 3D data is generated. The method of generating 3DPDF data also uses conventional art, so description will be omitted. InFIG. 33, this corresponds to the part denoted by 21603. FIGS. 34A and34B are display examples of opening the 3D PDF data using the PDFviewer. FIG. 34A is a display example of 3D PDF data 21700. As denotedby reference numeral 21701 in FIG. 34A, inserting the 3D data into thePDF data in step S21411 enables the 3D data to be viewed in the PDFviewer.

In step S21412, the 3D control unit 2417 of the medical image processingserver 2102 opens the 3D data file saved in step S21410 and places the3D data in three-dimensional space.

In step S21413, the display orientation changing unit 2422 of themedical image processing server 2102 changes the display orientation ofthe 3D data opened in step S21412 to one of the display orientationsstored in the display orientation 21504 in the commission informationbeing processed.

In step S21414, the 3D control unit 2417 of the medical image processingserver 2102 captures (takes a screenshot) of the 3D data of which thedisplay orientation has been changed in step S21413, and generates acapture image. Thus, a capture image is generated of the display part ofwhich the display orientation has been changed.

In step S21415, the 3D control unit 2417 of the medical image processingserver 2102 determines whether capturing has been performed for alldisplay orientations to be stored in the display orientation 21504 ofthe commission information being processed. The display orientation asused here includes the default display orientation as well. In a casewhere determination is made that this has been executed for all, theflow advances to step S21416. In a case where determination is made thatthis has not been executed for all, i.e., there remain displayorientations to be executed, the flow advances to step S21412.

In step S21416, the 3D PDF data generating unit 2421 of the medicalimage processing server 2102 inserts the capture image generated in stepS21414 into the 3D PDF data.

At this time, the capture image is inserted in a manner correlated withthe rotation axis 21002 and rotation angle 21003 of the displayorientation that the capture image assumes. The capture image(aforementioned object) is set to a state capable of accepting apressing operation. Thus, in a case where a selection of the captureimage has been accepted, the 3D data can be rotated to the displayorientation that the selected capture image indicates. Reference numeral21702 in FIG. 34A indicates the results of having inserted captureimages. Thus, 3D data of the display orientations that have beenselected is in the capture images, so the user can comprehend theresults of having changed the display orientation more intuitively thanof a text string indicating the display orientation is displayed. Thedefault display orientation also has a capture image generated, which isadvantageous in that the 3D PDF data does not need to be opened again toreturn the display orientation to the default display orientation.

In step S21417, the storage unit 2412 of the medical image processingserver 2102 saves the 3D PDF data generated in step S21416 in theexternal memory 2311 of the medical image processing server 2102. Thesaving location is then stored in the 3D PDF data saving location 21509of the commission information being processed.

In step S21418, the communication control unit 2411 of the medical imageprocessing server 2102 transmits processing results to the clientterminal 2101 to the effect that extracting the commission part andgenerating 3D PDF data has been successful. In step S21419, thecommunication control unit 2401 of the client terminal 2101 notifies theuser, who is the commissioning party.

Once processing up to step S21419 is completed, the client terminal 2101updates the medical image list screen 2700. That is to say, the seriesof processing illustrated in FIG. 22 is executed. Thus, the medicalimage list screen 2700 is updated to the newest state.

FIG. 35 is a flowchart diagram illustrating a series of processing ofconfirming commission results. The steps of step S21801, step S21802,step S21812, and step S21813 in FIG. 35 are processing executed by theCPU 2301 of the client terminal 2101. The steps of step S21803 throughstep S21811 in FIG. 35 are processing executed by CPU 2301 of themedical image processing server 2102. Note that the contents ofprocessing and the order of processing illustrated in FIG. 35 are onlyexemplary, and are not restrictive.

In step S21801, the operation accepting unit 2405 of the client terminal2101 determines whether or not a pressing operation of the confirmbutton 2703 of the medical image list screen 2700 has been accepted. Ina case where determination is made that a pressing operation of theconfirm button 2703 has been accepted, the flow advances to step S21802.In a case where determination is made that a pressing operation of theconfirm button 2703 has not been accepted, the flow stands by.

In step S21802, the Web browser unit 2403 of the client terminal 2101acquires the image ID embedded in the recorded of the confirm button2703 that has accepted the pressing operation. The communication controlunit 2401 of the client terminal 2101 then transmits an acquisitionrequest for confirmation results of the medical image that this image IDindicates (hereinafter referred to as “object medical image” in FIG.35), including this image ID, to the medical image processing server2102.

In step S21803, the communication control unit 2411 of the medical imageprocessing server 2102 receives the commission results acquisitionrequest transmitted from the client terminal 2101.

In step S21804, the storage unit 2412 of the medical image processingserver 2102 acquires the processing results of the object medical imagefrom the external memory 2311. The image ID of the object medical imagehas been received from the client terminal 2101, so the image ID 21502corresponding to this image ID is identified from the records in thecommission information management table 21500. The 3D data is thenacquired from the 3D data saving location 21508 of the identified record(hereinafter referred to as “commission information being processed” inthe description of FIG. 35).

In step S21805, the screen generating unit 2413 of the medical imageprocessing server 2102 acquires a template for a commission resultsconfirmation screen from the storage unit 2412, and the record on themedical image management table 2600 corresponding to the image ID 21502received in step S21804. These are then used to generate a commissionresults confirmation screen. The 3D data acquired in step S21804 isinserted into the generated commission results confirmation screen. Thetemplate of the commission results confirmation screen preferably is ofthe same layout as the 3D PDF data that has been generated. The terminsert may mean embedding 3D data into the screen, and may mean the 3Ddata being linked to from the screen.

FIGS. 36A and 36B illustrate display examples of a case of inserting the3D data into the commission results confirmation screen. FIG. 36A is adisplay example of a commission results confirmation screen 21900.Inserting the 3D data into the commission results confirmation screen instep S21805 enables the 3D data to be displayed in the Web browser. Afurther arrangement may be made, as denoted by reference numeral 21901in FIG. 36A, where a character string such as “SAMPLE” is superimposedon the 3D data, to prevent the user, who is the commissioning party,from capturing the 3D data at the client terminal 2101. The characterstring or an image is placed in a layer above the layer where the 3Ddata of the commission results confirmation screen 21900 is displayed,and the opacity is adjusted, to serve as a watermark. Alternatively, thecharacter string may be included in the 3D data.

In step S21806, the 3D control unit 2417 of the medical image processingserver 2102 opens the file of the 3D data acquired in step S21804, andplaces the 3D data in three-dimensional space.

In step S21807, the display orientation changing unit 2422 of themedical image processing server 2102 changes the display orientation ofthe 3D data opened in step S21806 to one of the display orientationsstored in the display orientation 21504 of the commission informationbeing processed identified in step S21804.

In step S21808, the 3D control unit 2417 of the medical image processingserver 2102 captures (takes a screenshot) of the 3D data of which thedisplay orientation has been changed in step S21807, and generates acapture image. Thus, a capture image is generated of the display part ofwhich the display orientation has been changed.

In step S21809, the 3D control unit 2417 of the medical image processingserver 2102 determines whether capturing has been performed for alldisplay orientations to be stored in the display orientation 21504 ofthe commission information being processed. The display orientation asused here includes the default display orientation as well. In a casewhere determination is made that this has been executed for all, theflow advances to step S21810. In a case where determination is made thatthis has not been executed for all, i.e., there remain displayorientations to be executed, the flow advances to step S21806.

In step S21810, the capture image insertion unit 2416 of the medicalimage processing server 2102 inserts the capture image generated in stepS21808 into the commission results confirmation screen 21900, so thatthe layout is the same as that of the generated 3D PDF data. At thistime, the capture image is inserted in a manner correlated with therotation axis 21002 and rotation angle 21003 of the display orientationthat the capture image assumes. The capture image (aforementionedobject) is set to a state capable of accepting a pressing operation.Thus, in a case where a selection of the capture image has beenaccepted, the 3D data can be rotated to the display orientation that thecapture image indicates. Reference numeral 21902 in FIG. 36A indicatesthe results of having inserted capture images. Thus, 3D data of thedisplay orientations that have been selected is in the capture images,so the user can comprehend the results of having changed the displayorientation more intuitively than of a text string indicating thedisplay orientation is displayed. The layout is the same as that of the3D PDF data in FIG. 34A, so the processing results are easier tocomprehend. The default display orientation also has a capture imagegenerated, which is advantageous in that the 3D PDF data does not needto be opened again to return the display orientation to the defaultdisplay orientation.

In step S21811, the communication control unit 2411 of the medical imageprocessing server 2102 transmits the commission results confirmationscreen 21900 generated in step S21810 to the client terminal 2101.

In step S21812, the communication control unit 2411 of the clientterminal 2101 receives the commission results confirmation screen 21900transmitted from the medical image processing server 2102. Thecommunication control unit 2401 receives the commission resultsconfirmation screen 21900 transmitted from the medical image processingserver 2102. The rendering unit 2404 renders the received commissionresults confirmation screen 21900, and the display unit 2402 displaysthe rendering results on the display 2310 of the client terminal 2101.

Thus, when having the commission results to be confirmed, the commissionresults confirmation screen is presented as a Web page having the samelayout as the 3D PDF data, and not as the 3D PDF data. This can preventthe user, who is the commissioning party, from just acquiring the 3D PDFdata without performing the processing of finalizing the confirmation,described later in FIG. 38.

Next, operations in the commission results confirmation screen 21900will be described. FIG. 37 is a flowchart diagram illustrating a seriesof processing executed in response to an operation accepted at thecommission results configuration screen 21900. The steps of step S22001through step S22005 in FIG. 37 are processing executed by CPU 2301 ofthe client terminal 2101. Note that the contents of processing and theorder of processing illustrated in FIG. 37 are only exemplary, and arenot restrictive. The series of processing illustrated in FIG. 37 isrepeatedly executed while the commission results confirmation screen21900 is being displayed.

In step S22001, the operation accepting unit 2405 of the client terminal2101 determines whether or not there has been a drag operation as to the3D data in the commission results confirmation screen 21900. Thecommission results confirmation screen 21900 has 3D data displayed bythe Web browser unit 2403 for the part selected as the display part, asindicated by reference numeral 21901, so determination is made regardingwhether or not a drag operation has been accepted thereon. In a casewhere determination is made that a drag operation has been accepted, theflow advances to step S22002. In a case where determination is made thata drag operation has not been accepted, or a different operation hasbeen accepted, the flow advances to step S22003.

In step S22002, the 3D data plugin unit 2406 of the client terminal 2101changes the display orientation of the 3D data in accordance with thedrag operation the moving amount and direction of movement of the dragoperation are acquired in real time, and the display orientation ischanged by rotating the 3D data accordingly. Thus, the user can confirmthe commission results.

In step S22003, the operation accepting unit 2405 of the client terminal2101 determines whether or not a selection (pressing) on a capture imagein the commission results confirmation screen 21900 has been accepted.

Capture images such as indicated by reference numeral 21902 are insertedinto the commission results confirmation screen 21900, so determinationis made regarding whether or not selection thereof has been accepted. Ina case where determination is made that selection has been maderegarding the capture image, the flow advances to step S22004. In a casewhere determination is made that selection has not been made regardingthe capture image, the series of processing ends.

In step S22004, the 3D data plugin unit 2406 of the client terminal 2101acquires the rotation axis and rotation angle corresponding to thecapture image regarding which the selection has been accepted. Thecaption image, and the display orientation that the capture image isassuming with regard to the rotation axis 1002 and rotation angle 1003are inserted in a correlated manner in the above-described step S21810,so this can be used.

In step S22005, the 3D data plugin unit 2406 of the client terminal 2101uses the rotation axis and rotation angle acquired in step S22004 tochange the display orientation of the 3D data displayed in thecommission results confirmation screen 21900. For example, when thecapture image indicating “bottom” in the commission results confirmationscreen 21900 in FIG. 36A is selected, the rotation axis and rotationangle (X axis, −90 degrees) corresponding to the capture image for“bottom” are acquired. The 3D data is rotated by the acquired rotationangle on the acquired rotation axis, thereby reaching the stateillustrated in FIG. 36B. Accordingly, the display orientation can bechanged to the same display orientation as the capture image, simply byselecting the capture image.

FIG. 38 is a flowchart diagram illustrating a series of processing offinalizing commissioning. The steps of step S22101, step S22102, stepS22108, and step S22109 in FIG. 38 are processing executed by the CPU2301 of the client terminal 2101. The steps of step S22103 through stepS22107 in FIG. 38 are processing executed by CPU 2301 of the medicalimage processing server 2102. Note that the contents of processing andthe order of processing illustrated in FIG. 38 are only exemplary, andare not restrictive.

In step S22101, the operation accepting unit 2405 of the client terminal2101 determines whether or not a pressing operation to the finalizebutton 2704 in the medical image list screen 2700 has been accepted. Ina case where determination is made that a pressing operation to thefinalize button 2704 has been accepted, the flow advances to stepS22102. In a case where determination is made that a pressing operationhas not been accepted, the flow stands by.

In step S22102, the Web browser unit 2403 of the client terminal 2101acquires the image ID embedded in the recorded of the finalize button2704 that has accepted the pressing operation. The communication controlunit 2401 of the client terminal 2101 then transmits a finalfinalization request for the medical image that this image ID indicates(hereinafter referred to as “object medical image” in FIG. 38), to themedical image processing server 2102.

In step S22103, the communication control unit 2411 of the medical imageprocessing server 2102 receives the finalization request transmittedfrom the client terminal 2101.

In step S22104, the storage unit 2412 of the medical image processingserver 2102 identifies the commission information including the image IDincluded in the finalization request received in step S22103, from thecommission information management table 21500. The generated 3D PDF datais acquired from the 3D PDF data saving location 21509 of the commissioninformation that has been identified.

In step S22105, the communication control unit 2411 of the medical imageprocessing server 2102 transmits the 3D PDF data acquired in step S22104to the client terminal 2101.

In step S22106, the medical image processing server 2102 issues aninvoice to the user, who is the commissioning party, in accordance withthe commission contents that have been accepted. In step S22107, thestorage unit 2412 of the medical image processing server 2102 updatesthe status 21507 of the commission information identified in step S22104to billing. Once the fee is paid thereafter, the status is changed tocommission completed, and the series of the commission ends.

On the other hand, in step S22108, the communication control unit 2401of the client terminal 2101 receives the 3D PDF data transmitted fromthe medical image processing server 2102. In step S22109, the PDF viewerunit 2407 of the client terminal 2101 saves the received 3D PDF data andthen opens it to be displayed on the display 2310 of the client terminal2101.

Next, operations of the 3D PDF data displayed by the PDF viewer unit2407 will be described. FIG. 39 is a flowchart diagram illustrating aseries of processing executed in response to an operation accepted atthe 3D PDF data. The steps of step S22201 through step S22205 in FIG. 39are processing executed by the CPU 2301 of the client terminal 2101.Note that the contents of processing and the order of processingillustrated in FIG. 39 are only exemplary, and are not restrictive. Theseries of processing illustrated in FIG. 39 is repeatedly executed whilethe 3D PDF data is being displayed.

In step S22201, the PDF viewer unit 2407 of the client terminal 2101determines whether or not there has been a drag operation as to the 3Ddata in the 3D PDF data. The 3D PDF data has 3D data displayed by thePDF viewer unit 2407 for the part selected as the display part, asindicated by reference numeral 21701, so determination is made regardingwhether or not a drag operation has been accepted thereon. In a casewhere determination is made that a drag operation has been accepted, theflow advances to step S22202. In a case where determination is made thata drag operation has not been accepted, or a different operation hasbeen accepted, the flow advances to step S22203.

In step S22202, the 3D data control unit 2408 of the client terminal2101 changes the display orientation of the 3D data in accordance withthe drag operation. The moving amount and direction of movement of thedrag operation are acquired in real time, and the display orientation ischanged by rotating the 3D data accordingly. Thus, the user can confirmthe commission results.

In step S22203, the PDF viewer unit 2407 of the client terminal 2101determines whether or not a selection (pressing) on a capture image inthe 3D PDF data has been accepted.

Capture images such as indicated by reference numeral 21702 are insertedinto the 3D PDF data, so determination is made regarding whether or notselection thereof has been accepted. In a case where determination ismade that selection has been made regarding the capture image, the flowadvances to step S22204. In a case where determination is made thatselection has not been made regarding the capture image, the series ofprocessing ends.

In step S22204, the 3D data control unit 2408 of the client terminal2101 acquires the rotation axis and rotation angle corresponding to thecapture image regarding which the selection has been accepted. Thecaption image, and the display orientation that the capture image isassuming with regard to the rotation axis 21002 and rotation angle 21003are inserted in a correlated manner in the above-described step S21416,so this can be used.

In step S22205, the 3D data control unit 2408 of the client terminal2101 uses the rotation axis and rotation angle acquired in step S22204to change the display orientation of the 3D data displayed in the 3D PDFdata. For example, when the capture image indicating “bottom” in the 3DPDF data in FIG. 34A is selected, the rotation axis and rotation angle(X axis, −90 degrees) corresponding to the capture image for “bottom”are acquired. The 3D data is rotated by the acquired rotation angle onthe acquired rotation axis, thereby reaching the state illustrated inFIG. 34B. Accordingly, the display orientation can be changed to thesame display orientation as the capture image, simply by selecting thecapture image. Also, the 3D data can be operated with the sameoperational feeling as the commission results confirmation screen 21900described above.

FIG. 40 is a flowchart diagram illustrating a series of processing ofcancelling commissioning. The steps of step S22301 and step S22302 inFIG. 40 are processing executed by the CPU 2301 of the client terminal2101. The steps of step S22303 through step S22305 in FIG. 40 areprocessing executed by CPU 2301 of the medical image processing server2102. Note that the contents of processing and the order of processingillustrated in FIG. 40 are only exemplary, and are not restrictive.

In step S22301, the operation accepting unit 2405 of the client terminal2101 determines whether or not a pressing operation to the cancel button2705 in the medical image list screen 2700 has been accepted. In a casewhere determination is made that a pressing operation to the cancelbutton 2705 has been accepted, the flow advances to step S22302. In acase where determination is made that a pressing operation to the cancelbutton 2705 has not been accepted, the flow stands by.

In step S22302, the Web browser unit 2403 of the client terminal 2101acquires the image ID embedded in the recorded of the cancel button 2705that has accepted the pressing operation. The communication control unit2401 of the client terminal 2101 then transmits a generation commissioncancel request for the medical image that this image ID indicates(hereinafter referred to as “object medical image” in FIG. 40),including this image ID, to the medical image processing server 2102.

In step S22303, the communication control unit 2411 of the medical imageprocessing server 2102 receives the cancel request transmitted from theclient terminal 2101. In step S22304, the storage unit 2412 of themedical image processing server 2102 deletes the commission results ofthe commission regarding which the cancel request has been made from theexternal memory 2311. The image ID has been received from the clientterminal 2101, so the image ID 21502 including corresponding to thisimage ID is identified from the record in the commission informationmanagement table 21500. The 3D data and 3D PDF data, which are thecommission results, are deleted from the identified 3D data savinglocation 21508 and 3D PDF data saving location 21509.

In step S22305, the storage unit 2412 of the medical image processingserver 2102 deletes the commission information regarding which thecancel request has been made, i.e., the record identified in stepS22304, from the commission information management table 21500. Thus thegeneration commission of 3D PDF data is canceled.

When the processing up to step S22305 is completed, the medical imageprocessing server 2102 transmits an update request of the medical imagelist screen 2700 to the client terminal 2101. Upon receiving this updaterequest, the client terminal 2101 executes the series of processingillustrated in FIG. 22. Accordingly, the medical image list screen 2700is updated to the newest state.

As described above, document data including three-dimensional dataindicating human body parts, generated from multiple medical images, canbe generated.

The present invention can be carried out as an embodiment in the form ofa system, apparatus, method, program or storage medium, or the like, forexample. Specifically, the present invention may be applied to a systemmade up of multiple devices, or to an apparatus made up of a singledevice.

Also, the present invention includes a computer-executable program of asoftware program to realize the functions of the above-describedembodiment, being directly or remotely supplied to a system orapparatus. The present invention can also be realized by the computer ofthe system or apparatus reading out and executing the supplied programcode.

Accordingly, in order to realize the function processing of the presentinvention on a computer, program code itself to be installed in thecomputer also can realize the present invention. That is to say, thepresent invention also includes the computer program itself forrealizing function processing of the present invention.

In this case, the mode of the program is irrelevant as long as there isa program function, and may be object code, a program executed by aninterpreter, script data supplied to an operating system (OS), or thelike.

Examples of a computer-readable storage medium for supplying the programinclude flexible disks, hard disks, optical disks (including such asCD-ROM, CD-R, CD-RW, DVD, DVD-ROM, and DVD-R), magneto-optical disks(MO), magnetic tape, non-volatile memory cards, ROM, and so forth.

Also, as a program supply method, the program can be supplied byaccessing a Web page on the Internet using a browser on the clientcomputer, and downloading the program to a storage medium such as a harddisk. In this case, the computer program itself of the present inventionmay be downloaded, or a compressed file including an automaticinstallation function may be downloaded.

Also, dividing the program code making up the program of the presentinvention into multiple files, and downloading each file from differenthome pages can also realize the present invention. That is to say, a WWWserver for downloading a program file to realize the function processingof the present invention by a computer as to multiple users is alsoincluded in the scope of the present invention.

Also, an arrangement may be made wherein the program of the presentinvention is encrypted, stored in a storage medium such as a CD-ROM, anddistributed to users who have cleared predetermined conditions, who candownload key information to decrypt the encryption from a Web page viathe Internet. The encrypted program can be decrypted and installed usingthe downloaded key information, so as to be installed to the computer.

Also, the computer executes the program that has been read out, wherebythe functions of the above-described embodiments can be realized. Forexample, an OS or the like running on the computer can perform part orall of the actual processing based on the program instructions, wherebythe above-described embodiment functions can be realized.

Further, an arrangement may be made wherein the program read from thestorage medium is loaded to memory provided to a function expansionboard inserted in the computer or a function expansion unit connected tothe computer. In this case, subsequently, a CPU or the like provided tothe function expansion board or function expansion unit may perform partor all of the actual processing based on the program instructions,whereby the above-described embodiment functions can be realized.

The above-described embodiments are only examples of specific ways tocarry out the present invention, and should not be interpreted asrestricting the technical scope of the present invention. That is tosay, the present invention can be carried out in various forms withoutdeparting from the technical spirit and primary features thereof.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application Nos.2015-084593 and 2015-084594 filed Apr. 16, 2015 and Japanese PatentApplication No. 2015-125953 filed Jun. 23, 2015, which are herebyincorporated by reference herein in their entirety.

What is claimed is:
 1. A medical image processing system that can acceptcommissions to perform 3D formation using a 3D forming apparatus, themedical image processing system comprising: a storage unit configured tostore a medical image, and body part information representing a humanbody part in the medical image; a selection accepting unit configured toaccept selection with regard to the medical image stored in the storageunit; a part identifying unit configured to identify the body partincluded in the medical image, using the body part informationcorresponding to the medical image regarding which the selectionaccepting unit has accepted selection; and a screen generating unitconfigured to generate a screen that can accept a commission to form aformed object of the body part identified by the part identifying unit.2. The medical image processing system according to claim 1, wherein thescreen generating unit further generates the screen such that formationcommissioning of a formed object representing a body part that was notidentified by the part identifying unit cannot be accepted.
 3. Themedical image processing system according to claim 1, furthercomprising: a formation commission accepting unit configured to accept,via the screen generated by the screen generating unit, a commission toform a formed object of the body part identified by the part identifyingunit; a three-dimensional data generating unit configured to generatethree-dimensional data of the body part, regarding which a formationcommission has been accepted by the formation commission accepting unit,using the medical image regarding which selection has been accepted atthe selection accepting unit; a formation job generating unit configuredto generate a formation job to form, using the 3D forming apparatus, aformed object of the three-dimensional data generated by thethree-dimensional data generating unit; and a formation job transmittingunit configured to transmit the formation job generated by the formationjob generating unit to the 3D forming apparatus.
 4. The medical imageprocessing system according to claim 3, wherein the screen generatingunit further generates the screen so as to be capable of acceptingselection of the formation method of the formed object, and wherein theformation job transmitting unit identifies the transmission destinationof the formation job in accordance with the formation method selectedvia the screen generated by the screen generating unit, and transmitsthe formation job to the identified 3D forming apparatus.
 5. A controlmethod for a medical image processing system that can accept commissionsto perform 3D formation using a 3D forming apparatus, wherein themedical image processing system includes a storage unit configured tostore a medical image, and body part information representing a humanbody part in the medical image, the control method comprising:accepting, via a selection accepting unit, selection with regard to themedical image stored in the storage unit; identifying the body partincluded in the medical image, using the body part informationcorresponding to the medical image regarding which the selectionaccepting unit has accepted selection; and generating a screen that canaccept a commission to form a formed object of the identified body part.6. A non-transitory storage medium storing a program to perform acontrol method for a medical image processing system that can acceptcommissions to perform 3D formation using a 3D forming apparatus,wherein the medical image processing system includes a storage unitconfigured to store a medical image, and body part informationrepresenting a human body part in the medical image, the control methodcomprising: accepting, via a selection accepting unit, selection withregard to the medical image stored in the storage unit; identifying thebody part included in the medical image, using the body part informationcorresponding to the medical image regarding which the selectionaccepting unit has accepted selection; and generating a screen that canaccept a commission to form a formed object of the identified body part.7. A medical image processing apparatus that can accept commissions toperform 3D formation using a 3D forming apparatus, the medical imageprocessing apparatus comprising: a storage unit configured to store amedical image, and body part information representing a human body partin the medical image; a selection accepting unit configured to acceptselection with regard to the medical image stored in the storage unit; apart identifying unit configured to identify the body part included inthe medical image, using the body part information corresponding to themedical image regarding which the selection accepting unit has acceptedselection; and a screen generating unit configured to generate a screenthat can accept a commission to form a formed object of the body partidentified by the part identifying unit.
 8. A control method for amedical image processing apparatus that can accept commissions toperform 3D formation using a 3D forming apparatus, wherein the medicalimage processing apparatus includes a storage unit configured to store amedical image, and body part information representing a human body partin the medical image, the control method comprising: accepting, via aselection accepting unit, selection with regard to the medical imagestored in the storage unit; identifying the body part included in themedical image, using the body part information corresponding to themedical image regarding which the selection accepting unit has acceptedselection; and generating a screen that can accept a commission to forma formed object of the identified body part.
 9. A non-transitory storagemedium storing a program to perform a control method for a medical imageprocessing apparatus that can accept commissions to perform 3D formationusing a 3D forming apparatus, wherein the medical image processingapparatus includes a storage unit configured to store a medical image,and body part information representing a human body part in the medicalimage, the control method comprising: accepting, via a selectionaccepting unit, selection with regard to the medical image stored in thestorage unit; identifying the body part included in the medical image,using the body part information corresponding to the medical imageregarding which the selection accepting unit has accepted selection; andgenerating a screen that can accept a commission to form a formed objectof the identified body part.
 10. A medical image processing system thatcan accept commissions to perform 3D formation using a 3D formingapparatus, the medical image processing system comprising: a storageunit configured to store medical images; and a screen generating unitconfigured to generate a screen that can accept a formation commissionusing the medical images stored in the storage unit, for each of themedical images, wherein the screen generating unit generates the screensuch that further formation commissions cannot be accepted regarding amedical image for which a formation commission has already been acceptedvia the screen generated by the screen generating unit.
 11. The medicalimage processing system according to claim 10, wherein the screengenerating unit generates the screen so as to include an acceptingportion for each medical image, wherein the accepting portion is capableof accepting a formation commission using the medical image stored inthe storage unit.
 12. The medical image processing system according toclaim 11, wherein the screen generating unit controls display of theaccepting portion corresponding a medical image regarding which aformation commission has already been accepted via the screen generatedby the screen generating unit, so that a further formation commissioncannot be performed as to that medical image.
 13. The medical imageprocessing system according to claim 10, wherein the storage unit storesa medical image count representing a quantity of medical images, andwherein the screen generating unit generates the screen so that, in acase where the medical image count stored in the storage unit is equalto or less than a predetermined count, a formation commission cannot becannot be accepted as to that medical image.
 14. A control method for amedical image processing system that can accept commissions to perform3D formation using a 3D forming apparatus, wherein the medical imageprocessing system includes a storage unit configured to store medicalimages, the control method comprising: generating a screen that canaccept a formation commission using the medical images stored in thestorage unit, for each of the medical images, wherein generatingincludes generating the screen such that further formation commissionscannot be accepted regarding a medical image for which a formationcommission has already been accepted via the generated screen.
 15. Anon-transitory recording medium storing a program to perform a controlmethod for a medical image processing system that can accept commissionsto perform 3D formation using a 3D forming apparatus, wherein themedical image processing system includes a storage unit configured tostore medical images, the control method comprising: generating a screenthat can accept a formation commission using the medical images storedin the storage unit, for each of the medical images, wherein generatingincludes generating the screen such that further formation commissionscannot be accepted regarding a medical image for which a formationcommission has already been accepted via the generated screen.
 16. Amedical image processing apparatus that can accept commissions toperform 3D formation using a 3D forming apparatus, the medical imageprocessing apparatus comprising: a storage unit configured to storemedical images; and a screen generating unit configured to generate ascreen that can accept a formation commission using the medical imagesstored in the storage unit, for each of the medical images, wherein thescreen generating unit generates the screen such that further formationcommissions cannot be accepted regarding a medical image for which aformation commission has already been accepted via the screen generatedby the screen generating unit.
 17. A control method for a medical imageprocessing apparatus that can accept commissions to perform 3D formationusing a 3D forming apparatus, wherein the medical image processingapparatus includes a storage unit configured to store medical images,the control method comprising: generating a screen that can accept aformation commission using the medical images stored in the storageunit, for each of the medical images, wherein generating includesgenerating the screen such that further formation commissions cannot beaccepted regarding a medical image for which a formation commission hasalready been accepted via the generated screen.
 18. A non-transitoryrecording medium storing a program to perform a control method for amedical image processing apparatus that can accept commissions toperform 3D formation using a 3D forming apparatus, wherein the medicalimage processing apparatus includes a storage unit configured to storemedical images, the control method comprising: generating a screen thatcan accept a formation commission using the medical images stored in thestorage unit, for each of the medical images, wherein generatingincludes generating the screen such that further formation commissionscannot be accepted regarding a medical image for which a formationcommission has already been accepted via the generated screen.
 19. Amedical image processing system that manages medical images, the medicalimage processing system comprising: a part identifying unit configuredto identify a body part included in a plurality of medical images; ascreen generating unit configured to generate a screen that can acceptselection of a body part identified by the part identifying unit; anaccepting unit configured to accept selection of the part via a screengenerated by the screen generating unit; a three-dimensional datagenerating unit configured to generate three-dimensional data expressingthe part accepted by the accepting unit, using the plurality of medicalimages; and a document data generating unit configured to generatedocument data including the three-dimensional data generated by thethree-dimensional data generating unit.
 20. The medical image processingsystem according to claim 19, wherein the accepting unit acceptsselection of a plurality of medical images, and wherein the partidentifying unit identifies a human body part included in the pluralityof medical images regarding which selection has been accepted by theaccepting unit.
 21. The medical image processing system according toclaim 19, wherein the document data generating unit generates thedocument data so that a display orientation of three-dimensional datagenerated by the three-dimensional data generating unit can be changed.22. The medical image processing system according to claim 19, whereinthe screen generating unit generates a screen that can accept selectionof the display orientation of three-dimensional data generated by thethree-dimensional data generating unit, and wherein the document datagenerating unit generates the document data so that the displayorientation of three-dimensional data generated by the three-dimensionaldata generating unit can be changed to the display orientation regardingwhich selection has been accepted via the screen.
 23. The medical imageprocessing system according to claim 19, wherein the screen generatingunit further generates a screen for confirmation of thethree-dimensional data generated by the three-dimensional datagenerating unit.
 24. A control method for a medical image processingsystem that manages medical images, the control method comprising:identifying a body part included in a plurality of medical images;generating a screen that can accept selection of an identified bodypart; accepting selection of the part via a generated screen; generatingthree-dimensional data expressing the accepted part, using the pluralityof medical images; and generating document data including the generatedthree-dimensional data.
 25. A non-transitory recording medium storing aprogram to perform a control method for a medical image processingsystem that manages medical images, the control method comprising:identifying a body part included in a plurality of medical images;generating a screen that can accept selection of an identified bodypart; accepting selection of the part via a generated screen; generatingthree-dimensional data expressing the accepted part, using the pluralityof medical images; and generating document data including the generatedthree-dimensional data.
 26. A medical image processing apparatus thatmanages medical images, the medical image processing apparatuscomprising: a part identifying unit configured to identify a body partincluded in a plurality of medical images; a screen generating unitconfigured to generate a screen that can accept selection of a body partidentified by the part identifying unit; an accepting unit configured toaccept selection of the part via a screen generated by the screengenerating unit; a three-dimensional data generating unit configured togenerate three-dimensional data expressing the part accepted by theaccepting unit, using the plurality of medical images; and a documentdata generating unit configured to generate document data including thethree-dimensional data generated by the three-dimensional datagenerating unit.
 27. A control method for a medical image processingapparatus that manages medical images, the control method comprising:identifying a body part included in a plurality of medical images;generating a screen that can accept selection of an identified bodypart; accepting selection of the part via a generated screen; generatingthree-dimensional data expressing the accepted part, using the pluralityof medical images; and generating document data including the generatedthree-dimensional data.
 28. A non-transitory recording medium storing aprogram to perform a control method for a medical image processingapparatus that manages medical images, the control method comprising:identifying a body part included in a plurality of medical images;generating a screen that can accept selection of an identified bodypart; accepting selection of the part via a generated screen; generatingthree-dimensional data expressing the accepted part, using the pluralityof medical images; and generating document data including the generatedthree-dimensional data.